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Abstract 

We present an efficient lock-free algorithm for parallel accessible hash 
tables with open addressing, which promises more robust performance 
and reliability than conventional lock-based implementations. "Lock-free" 
means that it is guaranteed that always at least one process completes 
its operation within a bounded number of steps. For a single processor 
architecture our solution is as efficient as sequential hash tables. On a 
multiprocessor architecture this is also the case when all processors have 
comparable speeds. The algorithm allows processors that have widely 
different speeds or come to a halt. It can easily be implemented using 
C-like languages and requires on average only constant time for insertion, 
deletion or accessing of elements. The algorithm allows the hash tables to 
grow and shrink when needed. 

Lock-free algorithms are hard to design correctly, even when appar- 
ently straightforward. Ensuring the correctness of the design at the ear- 
liest possible stage is a major challenge in any responsible system de- 
velopment. In view of the complexity of the algorithm, we turned to 
the interactive theorem prover PVS for mechanical support. We employ 
standard deductive verification techniques to prove around 200 invariance 
properties of our algorithm, and describe how this is achieved with the 
theorem prover PVS. 
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1 Introduction 



We are interested in efficient, reliable, parallel algorithms. The classical syn- 
chronization paradigm based on mutual exclusion is not most suited for this, 
since mutual exclusion often turns out to be a performance bottleneck, and fail- 
ure of a single process can force all other processes to come to a halt. This 
is the reason to investigate lock-free or wait-free concurrent objects, see e.g. 
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Lock-free and wait-free objects 

A concurrent object is an abstract data type that permits concurrent operations 
that appear to be atomic 1201 |2EJ . The easiest way to implement concurrent 
objects is by means of mutual exclusion, but this leads to blocking when the 
process that holds exclusive access to the object is delayed or stops functioning. 

The object is said to be lock-free if any process can be delayed at any point 
without forcing any other process to block and when, moreover, it is guaranteed 
that always some process will complete its operation in a finite number of steps, 
regardless of the execution speeds of the processes 0] E3 E0 121 GH| • The object 
is said to be wait-free when it is guaranteed that any process can complete any 
operation in a finite number of steps, regardless of the speeds of the other 
processes [§]. 

We regard "non-blocking" as synonymous to "lock-free" . In several recent 
papers, e.g. [2EJ, the term "non-blocking" is used for the first conjunct in 
the above definition of lock-free. Note that this weaker concept does not in 
itself guarantee progress. Indeed, without real blocking, processes might delay 
each other arbitrarily without getting closer to completion of their respective 
operations. The older literature 0EJE1 seems to suggest that originally "non- 
blocking" was used for the stronger concept, and lock-free for the weaker one. 
Be this as it may, we use lock-free for the stronger concept. 

Concurrent hash tables 

The data type of hash tables is very commonly used to efficiently store huge 
but sparsely filled tables. Before 2003, as far as we know, no lock-free algorithm 
for hash tables had been proposed. There were general algorithms for arbitrary 
wait-free objects [21 El El j but these are not very efficient. Furthermore, 
there are lock- free algorithms for different domains, such as linked lists |28| . 
queues [23 and memory management ^1 • 

In this paper we present a lock-free algorithm for hash tables with open 
addressing that is in several aspects wait-free. The central idea is that every 
process holds a pointer to a hash table, which is the current one if the process is 
not delayed. When the current hash table is full, a new hash table is allocated 
and all active processes join in the activity to transfer the contents of the current 
table to the new one. The consensus problem of the choice of a new table is 
solved by means of a test-and-set register. When all processes have left the 
obsolete table, it is deallocated by the last one leaving. This is done by means 
of a compare-and-swap register. Measures have been taken to guarantee that 
actions of delayed processes are never harmful. For this purpose we use counters 
that can be incremented and decremented atomically. 

After the initial design, it took us several years to establish the safety prop- 
erties of the algorithm. We did this by means of the proof assistant PVS [231 ■ 
Upon completion of this proof, we learned that a lock-free resizable hash table 
based on chaining was proposed in [25]. We come back to this below. 

Our algorithm is lock-free and some of the subtasks are wait-free. We allow 
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fully parallel insertion, assignment, deletion, and Ending of elements. Finding 
is wait-free, the other three are not. The primary cause is that the process 
executing it may repeatedly have to execute or join a migration of the hash 
table. Assignment and deletion are also not wait-free when other processes 
repeatedly assign to the same address successfully. 

Migration is called for when the current hash table is almost filled. This 
occurs when the table has to grow beyond its current upper bound, but also 
for maintenance after many insertions and deletions. The migration itself is 
wait-free, but, in principle, it is possible that a slow process is unable to access 
and use a current hash table since the current hash table is repeatedly replaced 
by faster processes. 

Migration requires subtle provisions, which can be best understood by con- 
sidering the following scenario. Suppose that process A is about to (slowly) 
insert an element in a hash table H\. Before this happens, however, a fast pro- 
cess B has performed migration by making a new hash table Hi, and copying 
the content from H\ to H^. If (and only if) process B did not copy the insertion 
of A, A must be informed to move to the new hash table, and carry out the 
insertion there. Suppose a process C comes into play also copying the content 
from Hi to H^. This must be possible, since otherwise B can stop copying, 
blocking all operations of other processes on the hash table, and thus violating 
the lock-free nature of the algorithm. Now the value inserted by A can but 
need not be copied by both B and/or C. This can be made more complex by 
a process D that attempts to replace H 2 by H 3 . Still, the value inserted by A 
should show up exactly once in the hash table, and it is clear that processes 
should carefully keep each other informed about their activities on the tables. 

Performance, comparison, and correctness 

For a single processor architecture our solution is of the same order of efficiency 
as sequential hash tables. Actually, only an extra check is required in the main 
loop of the main functions, one extra bit needs to be set when writing data 
in the hashtables and at some places a write operation has been replaced by a 
compare and swap, which is more expensive. For ordinary operations on the 
hashtable, this is the only overhead and therefore a linear speed up can be 
expected on multiprocessor systems. The only place where no linear speed up 
can be achieved is when copying the hashtable. Especially, when processes have 
widely different speeds, a logaritmic factor may come into play (see algorithms 
for the write all problem [71 llfij V Indeed, initial experiments indicate that our 
algorithm is as efficient as sequential hash tables. It seems to require on average 
only constant time for insertion, deletion or accessing of elements. 

Some differences between our algorithm and the algorithm of |25j are clear. 
In our algorithm, the hashed values need not be stored in dynamic nodes if 
the address-value pairs (plus one additional bit) fit into one word. Our hash 
table can shrink whereas the table of bucket headers in [251 cannot shrink. A 
disadvantage of our algorithm, due to its open addressing, is that migration is 
needed as maintenance after many insertions and deletions. 
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An apparent weakness of our algorithm is the worst-case space complexity 
in the order of 0{PM) where P is the number of processes and M is the size 
of the table. This only occurs when many of the processes fail or fall asleep 
while using the hash table. Failure while using the hash table can be made less 
probable by adequate use of procedure "releaseAccess" . This gives a trade-off 
between space and time since it introduces the need of a corresponding call of 
"getAccess" . When all processes make ordinary progress and the hash table is 
not too small, the actual memory requirement in 0(M). 

The migration activity requires worst-case 0(M 2 ) time for each participating 
process. This only occurs when the migrating processes tend to choose the 
same value to migrate and the number of collisions is 0(M) due to a bad hash 
function. This is costly, but even this is in agreement with wait-freedom. The 
expected amount of work for migration for all processes together is 0(M) when 
collisions are sparse, as should be the case when migrating to a hash table that 
is sufficiently large. 

A true problem of lock-free algorithms is that they are hard to design cor- 
rectly, which even holds for apparently straightforward algorithms. Whereas 
human imagination generally suffices to deal with all possibilities of sequential 
processes or synchronized parallel processes, this appears impossible (at least to 
us) for lock- free algorithms. The only technique that we see fit for any but the 
simplest lock-free algorithms is to prove the correctness of the algorithm very 
precisely, and to verify this using a proof checker or theorem prover. 

As a correctness notion, we take that the operations behave the same as for 
'ordinary' hash tables, under some arbitrary linearization of these opera- 
tions. So, if a find is carried out strictly after an insert, the inserted element is 
found. If insert and find are carried out at the same time, it may be that find 
takes place before insertion, and it is not determined whether an element will 
be returned. 

Our algorithm contains 81 atomic statements. The structure of our algo- 
rithm and its correctness properties, as well as the complexity of reasoning 
about them, makes neither automatic nor manual verification feasible. We have 
therefore chosen the higher-order interactive theorem prover PVS for me- 

chanical support. PVS has a convenient specification language and contains a 
proof checker which allows users to construct proofs interactively, to automati- 
cally execute trivial proofs, and to check these proofs mechanically. 

Overview of the paper 

Section [21 contains the description of the hash table interface offered to the 
users. The algorithm is presented in Section|3J Section 0] contains a description 
of the proof of the safety properties of the algorithm: functional correctness, 
atomicity, and absence of memory loss. This proof is based on a list of around 
200 invariants, presented in Appendix A, while the relationships between the 
invariants are given by a dependency graph in Appendix B. Progress of the 
algorithm is proved informally in Section [S] Conclusions are drawn in Section 

ED 
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2 The interface 

The aim is to construct a hash table that can be accessed simultaneously by 
different processes in such a way that no process can passively block another 
process' access to the table. 

A hash table is an implementation of (partial) functions between two do- 
mains, here called Address and Value. The hash table thus implements a mod- 
ifiable shared variable X : Address — > Value. The domains Address and Value 
both contain special default elements <G Address and null G Value. An equal- 
ity X(a) = null means that no value is currently associated with the address 
a. In particular, since we never store a value for the address 0, we impose the 
invariant 

X(0) = null . 

We use open addressing to keep all elements within the table. For the imple- 
mentation of the hash table we require that from every value the address it 
corresponds to is derivable. We therefore assume that some function ADR : 
Value — > Address is given with the property that 

Axl: v = null = ADR(v) = 

Indeed, we need null as the value corresponding to the undefined addresses and 
use address as the (only) address associated with the value null. We thus 
require the hash table to satisfy the invariant 

X(a) ^ null => ADR(X(a)) = a . 

Note that the existence of ADR is not a real restriction since one can choose to 
store the pair (a, v) instead of v. When a can be derived from v, it is preferable 
to store v, since that saves memory. 

There are four principle operations: find, delete, insert and assign. The first 
one is to find the value currently associated with a given address. This operation 
yields null if the address has no associated value. The second operation is to 
delete the value currently associated with a given address. It fails if the address 
was empty, i.e. X(a) = null. The third operation is to insert a new value for a 
given address, provided the address was empty. So, note that at least one out 
of two consecutive inserts for address a must fail, except when there is a delete 
for address a in between them. The operation assign does the same as insert, 
except that it rewrites the value even if the associated address is not empty. 
Moreover, assign never fails. 

We assume that there is a bounded number of processes that may need to 
interact with the hash table. Each process is characterized by the sequence of 
operations 

( getAccess ; (find + delete + insert + assign)* ; releaseAccess)" 

A process that needs to access the table, first calls the procedure getAccess to 
get the current hash table pointer. It may then invoke the procedures find, 
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delete, insert, and assign repeatedly, in an arbitrary, serial manner. A process 
that has access to the table can call releaseAccess to log out. The processes may 
call these procedures concurrently. The only restriction is that every process 
can do at most one invocation at a time. 

The basic correctness conditions for concurrent systems are functional cor- 
rectness and atomicity, say in the sense of [201 ? Chapter 13. Functional correct- 
ness is expressed by prescribing how the procedures find, insert, delete, assign 
affect the value of the abstract mapping X in relation to the return value. Atom- 
icity means that the effect on X and the return value takes place atomically at 
some time between the invocation of the routine and its response. Each of these 
procedures has the precondition that the calling process has access to the table. 
In this specification, we use auxiliary private variables declared locally in the 
usual way. We give them the suffix S to indicate that the routines below are 
the specifications of the procedures. We use angular brackets ( and ) to indicate 
atomic execution of the enclosed command. 

proc finds (a : Address \ {0}) : VaJue = 
local rS : Value; 
(fS) < rS := X(a) }; 

return rS. 

proc deletes (a : Address \ {0}) : Bool = 
local sucS : Bool; 
(dS) ( sucS := (X(a) ^ null) ; 

if sucS then X(a) := null end } ; 
return sucS. 

proc inserts (v : Vk/ue \ {null}) : Bool = 

local sucS : Bool ; a : Address :— ADR(v) ; 
(iS) ( sucS := (X(a) = null) ; 

if sucS then X(a) := v end ) ; 
return sucS. 

proc assign s (v : Value \ {null}) = 
local a : Address := ADR(v) ; 
(aS) ( X(a) := v ) ; 

end. 

Note that, in all cases, we require that the body of the procedure is executed 
atomically at some moment between the beginning and the end of the call, but 
that this moment need not coincide with the beginning or end of the call. This 
is the reason that we do not (e.g.) specify find by the single line return X(a). 

Due to the parallel nature of our system we cannot use pre and postcondi- 
tions to specify it. For example, it may happen that insert(v) returns true while 
X(ADR(v)) v since another process deletes ADR(v) between the execution of 
(iS) and the response of insert. 

In Section EH 

we provide implementations for the operations find, delete, 
insert, assign. We prove partial correctness of the implementations by extending 
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them with the auxiliary variables and commands used in the specification. So, 
we regard X as a shared auxiliary variable and rS and sucS as private auxiliary 
variables; we augment the implementations of End, delete, insert, assign with 
the atomic commands (fS), (dS), (iS), (aS), respectively. We prove that each 
of the four implementations executes its specification command always exactly 
once and that the resulting value r or sue of the implementation equals the 
resulting value rS or sucS in the specification. It follows that, by removing the 
implementation variables from the combined program, we obtain the specifica- 
tion. This removal may eliminate many atomic steps of the implementation. 
This is known as removal of stutterings in TLA or abstraction from r steps 
in process algebras. 

3 The algorithm 

An implementation consists of P processes along with a set of variables, for 
P > 1. Each process, numbered from 1 up to P, is a sequential program 
comprised of atomic statements. Actions on private variables can be added to 
an atomic statement, but all actions on shared variables must be separated into 
atomic accesses. Since auxiliary variables are only used to facilitate the proof 
of correctness, they can be assumed to be touched instantaneously without 
violation of the atomicity restriction. 

3.1 Hashing 

We implement function X via hashing with open addressing, cf. |171 125] . We do 
not use direct chaining, where colliding entries are stored in a secondary list, as 
is done in j2S] . A disadvantage of open addressing with deletion of elements is 
that the contents of the hash table must regularly be refreshed by copying the 
non-deleted elements to a new hash table. As we wanted to be able to resize 
the hash tables anyhow, we consider this less of a burden. 

In principle, hashing is a way to store address-value pairs in an array (hash 
table) with a length much smaller than the number of potential addresses. The 
indices of the array are determined by a hash function. In case the hash function 
maps two addresses to the same index in the array there must be some method 
to determine an alternative index. The question how to choose a good hash 
function and how to find alternative locations in the case of open addressing is 
treated extensively elsewhere, e.g. |T7) . 

For our purposes it is convenient to combine these two roles in one abstract 
function key given by: 

key(a : Address, I : Nat, n : Nat) : Nat , 

where I is the length of the array (hash table), that satisfies 

Ax2: <key{a,l,n) <l 
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for all a, I, and n. The number n serves to obtain alternative locations in case of 
collisions: when there is a collision, we re-hash until an empty "slot" (i.e. null) 
or the same address in the table is found. The approach with a third argument 
n is unusual but very general. It is more usual to have a function Key dependent 
on a and I, and use a second function Inc, which may depend on a and I, to use 
in case of collisions. Then our function key is obtained recursively by 

key (a, 1, 0) = Key(a, I) and key(a, I, n + 1) = Inc(a, /, key(a, I, n)) . 

We require that, for any address a and any number I, the first I keys are all 
different, as expressed in 

Ax3: < k < m < I => key(a, I, k) ^ key(a, I, m) . 

3.2 Tagging of values 

As is well known ^7], hashing with open addressing needs a special value del G 
Value to replace deleted values. 

When the current hash table becomes full, the processes need to reach con- 
sensus to allocate a new hash table of new size to replace the current one. Then 
all values except null and del must be migrated to the new hash table. A value 
that is being migrated cannot be simply removed, since the migrating process 
may stop functioning during the migration. Therefore, a value being copied 
must be tagged in such a way that it is still recognizable. This is done by the 
function old. We thus introduce an extended domain of values to be called 
EValue, which is defined as follows: 

EValue = {del} U Value U {oid(w) | v G Value} 

We furthermore assume the existence of functions val : EValue — » Vaiue and 
oldp : EValue — > Bool that satisfy, for all v G Vaiue: 

val(v) = v oldp(v) = false 

vai(del) = null oWp(del) = false 
val(old{v)) = v oldp(old(v)) = true 

Note that the old tag can easily be implemented by designating one special bit 
in the representation of Vaiue. In the sequel we write done for oid(null). More- 
over, we extend the function ADR to domain EVaiue by ADR(v) = ADR(val(v)). 

3.3 Data structure 

A Hash table is either _L, indicating the absence of a hash table, or it has the 
following structure: 

size, bound, occ, dels : Nat] 
table : array . . size-1 of EValue. 
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The field size indicates the size of the hash table, bound the maximal number 
of places that can be occupied before refreshing the table. Both are set when 
creating the table and remain constant. The variable occ gives the number of 
occupied positions in the table, while the variable dels gives the number of 
deleted positions. If h is a pointer to a hash table, we write ft,. size, h.occ, 
h. dels and h. bound to access these fields of the hash table. We write ft,.table[i] 
to access the i th EValue in the table. 

Apart from the current hash table, which is the main representative of the 
variable X, we have to deal with old hash tables, which were in use before the 
current one, and new hash tables, which can be created after the current one. 

We now introduce data structures that are used by the processes to find and 
operate on the hash table and allow to delete hash tables that are not used 
anymore. The basic idea is to count the number of processes that are using a 
hash table, by means of a counter busy. The hash table can be thrown away 
when busy is set to 0. An important observation is that busy cannot be stored 
as part of the hash table, in the same way as the variables size, occ and bound 
above. The reason for this is that a process can attempt to access the current 
hash table by increasing its busy counter. However, just before it wants to write 
the new value for busy it falls asleep. When the process wakes up the hash table 
might have been deleted and the process would be writing at a random place in 
memory. 

This forces us to use separate arrays H and busy to store the pointers to hash 
tables and the busy counters. There can be 2P hash tables around, because 
each process can simultaneously be accessing one hash table and attempting to 
create a second one. The arrays below are shared variables. 

H : array 1 . . 2P of pointer to Hashtable ; 
busy : array 1 . . 2P of Nat ; 
prot : array 1 . . 2P of Nat ; 
next : array 1 . . 2P of . . 2P . 

As indicated, we also need arrays prot and next. The variable next[i] points 
to the next hash table to which the contents of hash table H[i] is being copied. 
If next[i] equals 0, this means that there is no next hash table. The variable 
prot[z] is used to guard the variables busy[z], next[i] and H[i] against being 
reused for a new table, before all processes have discarded them. 

We use a shared variable currlnd to hold the index of the currently valid 
hash table: 

currlnd : 1 . . 2P . 

Note however that after a process copies currlnd to its local memory, other 
processes may create a new hash table and change currlnd to point to that 
one. 

It is assumed that initially H[l] is pointing to some hash table. The other 
initial values of the shared variables are given by 
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currlnd = busyfl] = prot[l] = 1 , 

H[i] = busy[j] = prot[z] = for alii ^ 1 , 

next[i] = for all i. 

3.4 Primary procedures 

We first provide the code for the primary procedures, which match directly with 
the procedures in the interface. Every process has a private variable 

index : 1 . . 2P; 

containing what it regards as the currently active hash table. At entry of each 
primary procedure, it must be the case that the variable H[index] contains valid 
information. In section 13.51 we provide procedure getAccess with the main 
purpose to guarantee this property. When getAccess has been called, the system 
is obliged to keep the hash table at index stored in memory, even if there are 
no accesses to the hash table using any of the primary procedures. A procedure 
releaseAccess is provided to release resources, and it should be called whenever 
the process will not access the hash table for some time. 

3.4.1 Syntax 

We use a syntax analogous to Modula-3 [B]. We use := for the assignment. We 
use the C-operations ++ and — for atomic increments and decrements. The 
semicolon is a separator, not a terminator. The basic control mechanisms are 

loop .. end is an infinite loop, terminated by exit or return 
while .. do .. end and repeat .. until .. are ordinary loops 
if .. then .. {elsif ..} [else ..] end is the conditional 
case .. end is a case statement. 

Types are slanted and start with a capital. Shared variables and shared data 
elements are in typewriter font. Private variables are slanted or in math italic. 

3.4.2 The main loop 

We model the clients of the hash table in the following loop. This is not an 
essential part of the algorithm, but it is needed in the PVS description, and 
therefore provided here. 

loop 

0: getAccessQ ; 

loop 

1: choose call; case call of 

(/, a) with d^O^ Gnd(a) 
(d, a) with a/0-» delete(a) 
(i, v) with v 7^ null — > insert(v) 
(a, v) with v null — > assign(v) 
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(r) — ► reieaseAccess(index); exit 



end 



end 



end 



The main loop shows that each process repeatedly invokes its four principle 
operations with correct arguments in an arbitrary, serial manner. Procedure 
getAccess has to provide the client with a protected value for index. Procedure 
releaseAccess releases this value and its protection. Note that exit means a 
jump out of the inner loop. 

3.4.3 Procedure find 

Finding an address in a hash table with open addressing requires a linear search 
over the possible hash keys until the address or an empty slot is found. The 
kernel of procedure find is therefore: 



The main complication is that, when the process encounters an entry done (i.e. 
oW(null)), it has to join the migration activity by calling refresh. 

Apart from a number of special commands, we group statements such that 
at most one shared variable is accessed and label these 'atomic' statements with 
a number. The labels are chosen identical to the labels in the PVS code, and 
therefore not completely consecutive. 

In every execution step, one of the processes proceeds from one label to a 
next one. The steps are thus treated as atomic. The atomicity of steps that refer 
to shared variables more than once is emphasized by enclosing them in angular 
brackets. Since procedure calls only modify private control data, procedure 
headers are not always numbered themselves, but their bodies usually have 
numbered atomic statements. 



n := ; 

repeat r := h. table [key(a, I, n)] ; n++ ; 
until r = null V a = ADR(r) ; 



5: 
6: 



proc hnd(a : Address \ {0}) : Value = 

local r : EValue ; n,l : Nat ; h : pointer to Hashtable ; 
h := H[index] ; n := ; {cnt := 0} ; 
I := /i.size ; 
repeat 



7: 



( r := h.table[key(a,l,n)} ; 



{ if r = null V a = ADR(r) then cnt++ ; (fS) end } ) ; 



8 



10 
11 



if r = done then 

refreshQ ; 

h := H[index] ; n := ; 
I := /i.size ; 



else n++ end ; 



13 
14 



until r = null Va = ADR(r) ; 
return val(r) . 
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In order to prove correctness, we add between braces instructions that only 
modify auxiliary variables, like the specification variables X and rS and other 
auxiliary variables to be introduced later. The part between braces is comment 
for the implementation, it only serves in the proof of correctness. The private 
auxiliary variable cnt of type Nat counts the number of times (fS) is executed 
and serves to prove that (fS) is executed precisely once in every call of find. 

This procedure matches the code of an ordinary find in a hash table with 
open addressing, except for the code at the condition r = done. This code is 
needed for the case that the value at address a has been copied, in which case 
the new table must be located. Locating the new table is carried out by the 
procedure refresh, which is discussed in Section 1^31 In line 7, the accessed hash 
table should be valid (see invariants f]4 and He4 in Appendix A). After refresh 
the local variables n, h and I must be reset, to restart the search in the new 
hash table. If the procedure terminates, the specifying atomic command (fS) 
has been executed precisely once (see invariant Cnl) and the return values of 
the specification and the implementation are equal (see invariant Col). If the 
operation succeeds, the return value must be a valid entry currently associated 
with the given address in the current hash table. It is not evident but it has been 
proved that the linear search of the process executing find cannot be violated 
by other processes, i.e. no other process can delete, insert, or rewrite an entry 
associated with the same address (as what the process is looking for) in the 
region where the process has already searched. 

We require that every valid hash table contains at least one entry null or 
done. Therefore, the local variable n in the procedure find never goes beyond 
the size of the hash table (see invariants Cul, fi.4, fi.5 and axiom Ax2). When 
the bound of the new hash table is tuned properly before use (see invariants 
Ne7, Ne8), the hash table will not be updated too frequently, and termination 
of the procedure End can be guaranteed. 

3.4.4 Procedure delete 

To some extent, deletion is similar to finding. Since r is a local variable to the 
procedure delete, we regard 18a and 18b as two parts of atomic instruction 18. 
If the entry is found in the table, then at line 18b this entry is overwritten with 
the designated element del. 

proc delete(a : Address \ {0}) : Bool = 
local r : EValue ; k,l,n ; Nat ; 

h : pointer to Hashtable ; sue : Bool ; 
15: h := H[index] ; sue := faise ; {cnt := 0} ; 

16: I := /i.size ; n := ; 

repeat 

17: k := key(a, I, n) ; 

( r :— h.table[k] ; 
{ if r = null then cnt++ ; (dS) end } ) ; 
18a: if oldp(r) then 
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refreshQ ; 
20: h := E[indcx] ; 

21: I := h.size ; n := ; 

elsif a = ADB(r) then 
18b: ( if r = h.table[k] then 

sue := true ; ft.table[fc] := del ; 
{ cnt++ ; (dS) ; Y[k] := del } 
end ) ; 
else n++ end ; 
until sue Vr = null ; 
25: if sue then h.dels++ end ; 

26: return sue . 

The repetition in this procedure has two ways to terminate. Either deletion 
fails with r = null in 17, or deletion succeeds with r = /i.table[fc] in 18b. In 
the latter case, we have in one atomic statement a double access of the shared 
variable /i.table[fc]. This is a so-called compare&swap instruction. Atomicity is 
needed here to preclude interference. The specifying command (dS) is executed 
either in 17 or in 18b, and it is executed precisely once (see invariant Cn2), since 
in 18b the guard a = ADR(r) implies r ^ null (sec invariant del and axiom 
Axl). 

In order to remember the address from the value rewritten to done after the 
value is being copied in the procedure moveContents, in 18, we introduce a new 
auxiliary shared variable Y of type array of EValue, whose contents equals the 
corresponding contents of the current hash table almost everywhere except that 
the values it contains are not tagged as old or rewritten as done (see invariants 
Cu9, CulO). 

Since we postpone the increment of h.dels until line 25, the field dels is a 
lower bound of the number of positions deleted in the hash table (see invariant 
Cu4). 

3.4.5 Procedure insert 

The procedure for insertion in the table is given below. Basically, it is the 
standard algorithm for insertion in a hash table with open addressing. Notable 
is line 28 where the current process finds that the current hash table too full, 
and orders a new table to be made. We assume that h. bound is a number less 
than /i.size (see invariant Cu3), which is tuned for optimal performance. 

Furthermore, in line 35, it can be detected that values in the hash table have 
been marked old, which is a sign that hash table h is outdated, and the new 
hash table must be located to perform the insertion. 

proc insert(v : Value \ {null}) : Bool = 

local r : EValue ; k,l,n : Nat ; h : pointer to Hashtable ; 
sue : Bool ; a : Address := ADR(v) ; 
27: h := R[indcx] ; {cut := 0} ; 

28: if ft.occ > ft,. bound then 
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newTableQ ; 
30: h := H[index] end ; 

31: n := ; I := /i.size ; sue :— false ; 

repeat 

32: k := kcy(a, I, n) ; 

33: ( r := ft.table[fc] ; 

{ if a = ADR(r) then cnt++ ; (iS) end } } ; 
35a: if oldp(r) then 

refreshQ ; 
36: h := H [index] ; 

37: n := ; I := ft,. size ; 

elsif r = null then 
35b: ( if ft.table[fe] = null then 

sue := true ; ft.table[fc] := v ; 
{ cnt++ ; (iS) ; Y[k] := w } 
end ) ; 
else n++ end ; 
until sue Va = ADR(r) ; 
41: if sue then /i.occ++ end ; 

42: return sue . 

Instruction 35b is a version of compare&swap. Procedure insert terminates 
successfully when the insertion to an empty slot is completed, or it fails when 
there already exists an entry with the given address currently in the hash table 
(see invariant Co3 and the specification of insert). 



3.4.6 Procedure assign 

Procedure assign is almost the same as insert except that it rewrites an entry 
with a given value even when the associated address is not empty. We provide 
it without further comments. 

proc assign(v : Value \ {null}) = 

local r : EValuc ; k,l,n: Nat ; h : pointer to Hashtable ; 
sue : Bool ; a : Address := ADR(v) ; 
43: h := H[index] ; cnt := 0; 

44: if kocc > h. bound then 

newTable() ; 
46: h := H [index] end ; 

47: n := ; I := /i.size ; sue := faise ; 

repeat 

48: k := key(a, I, n) ; 

49: r := /i.table[fc] ; 

50a: if oldp(r) then 

refreshQ ; 
51: h := H[index] ; 

52: n := ; I := h. size ; 
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elsif r = null V a = ADR(r) then 
50b: ( if ft.table[fe] = r then 

sue :— true ; h.table[k] := v ; 
{ cnt++ ; (aS) ; Y[k] := v } 
end ) 
else n++ end ; 
until sue ; 

57: if r = null then /i.occ++ end ; 

end. 



3.5 Memory management and concurrent migration 

In this section, we provide the public procedures getAccess and releaseAccess 
and the auxiliary procedures refresh and ncwTable which are responsible for al- 
location and deallocation. We begin with the treatment of memory by providing 
a model of the heap. 



3.5.1 The model of the heap 

We model the Heap as an infinite array of hash tables, declared and initialized 
in the following way: 

Heap : array Nat of Hashtablc := ([JVat]_L) ; 
H_ index : Nat := 1 . 

So, initially, Heap[i] = _L for all indices i. The indices of array Heap are the 
pointers to hash tables. We thus simply regard pointer to Hashtable as a 
synonym of Nat. Therefore, the notation h. table used elsewhere in the paper 
stands for Heap[/i]. table. Since we reserve (to be distinguished from the absent 
hash table _L and the absent value null) for the null pointer (i.e. Heap[0] = _L, 
see invariant Hcl), we initialize H_ index, which is the index of the next hash 
table, to be 1 instead of 0. Allocation of memory is modeled in 

proc allocate(s, b : Nat) : Nat = 

( Heap[H_ index] := blank hash table with size = s, bound = b, 
occ = dels = ; 
H_index++ ) ; 
return H index ; 

We assume that allocate sets all values in the hash table Heap[H_ index] to null, 
and also sets its fields size and bound as specified. The variables occ and dels 
are set to because the hash table is completely filled with the value null. 
Deallocation of hash tables is modeled by 

proc dcAlloc(h : Nat) = 

( assert Heap[/i] ^ _L ; Heap[/i] := _L ) 
end . 

The assert here indicates the obligation to prove that deAlloc is called only for 
allocated memory 
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3.5.2 Procedure getAccess 

The procedure getAccess is defined as follows. 

proc getAccess() = 
loop 



59: index := currlnd; 

60: prot [index] ++ ; 

61: if index — currlnd then 

62: busy[index]++ ; 

63: if index = currlnd then return ; 

else releaseAccess(index) end ; 

65: else prot[index] — end ; 



end 
end. 

This procedure is a bit tricky. When the process reaches line 62, the index has 
been protected not to be used for creating a new hash table in the procedure 
newTable (see invariants pr2, pr3 and nT12). 

The hash table pointer H [index] must contain the valid contents after the 
procedure getAccess returns (see invariants Ot3, He4). So, in line 62, busy is 
increased, guaranteeing that the hash table will not inadvertently be destroyed 
(see invariant bul and line 69). Line 63 needs to check the index again in case 
that instruction 62 has the precondition that the hash table is not valid. Once 
some process gets hold of one hash table after calling getAccess, no process can 
throw it away until the process releases it (see invariant rA7). 

3.5.3 Procedure releaseAccess 

The procedure refeaseAccess is given by 

proc releaseAccess (i : 1 . . 2P) = 



local h : pointer to Hashtable ; 
67: h := H[i] ; 

68: busy[i] — ; 

69: if 1^0 A busy[z] = then 

70: ( if = h then := ; ) 

71: deAlloc(h) ; 

end ; 

end ; 

72: prot[i] — ; 



end. 

The test h ^ at 69 is necessary since it is possible that h = at the lines 68 
and 69. This occurs e.g. in the following scenario. Assume that process p is 
at line 62 with index ^ currlnd, while the number i = index satisfies H[i] = 
and busy[i] = 0. Then process p increments busy[i], calls releaseAccess(i) , and 
arrives at 68 with h = 0. 
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Since dcAlloc in line 71 accesses a shared variable, we have separated its call 
from 70. The counter busy[i] is used to protect the hash table from premature 
deallocation. Only if busy[i]=0, H[i] can be released. The main problem of the 
design at this point is that it can happen that several processes concurrently 
execute releaseAccess for the same value of i, with interleaving just after the 
decrement of busy[z]. Then they all may find busy[i] = 0. Therefore, a bigger 
atomic command is needed to ensure that precisely one of them sets H[i] to 
(line 70) and calls deAlloc. Indeed, in line 71, dcAlloc is called only for 
allocated memory (see invariant rA3). The counter prot[i] can be decreased 
since position i is no longer used by this process. 

3.5.4 Procedure newTable 

When the current hash table has been used for some time, some actions of the 
processes may require replacement of this hash table. Procedure newTable is 
called when the number of occupied positions in the current hash table exceeds 
the bound (see lines 28, 44). Procedure newTable tries to allocate a new hash 
table as the successor of the current one. If several processes call newTable 
concurrently, they need to reach consensus on the choice of an index for the 
next hash table (in line 84). A newly allocated hash table that will not be used 
must be deallocated again. 

proc newTablcQ = 

local i : 1 . . 2P ; b, bb : Bool ; 
77: while next [index] = do 

78: choose i e 1 . . 2P ; 

( b := (prot[i] - 0) ; 

if b then prot[i] := 1 end ) ; 
if b then 
81: busy[i] := 1 ; 

82: choose bound > H [index]. bound — H[index].dels + 2P ; 

choose size > bound + 2P ; 
E[i] := allocate (size, bound) ; 
83: next[i] := ; 

84: ( bb := (next[indcx] = 0) ; 

if bb then next [index] := i end ) ; 
if ^bb then releaseAccess(i) end ; 
end end ; 
refreshQ ; 
end . 

In command 82, we allocate a new blank hash table (see invariant nT8), of which 
the bound is set greater than H [index] .bound — H[index].dels + 2P in order to 
avoid creating a too small hash table (see invariants nT6, nT7). 

We require the size of a hash table to be more than bound + 2P because 
of the following scenario: P processes find "/i.occ > h. bound" at line 28 and 
call newtable, refresh, migrate, moveContents and moveElement one after the 
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other. After moving some elements, all processes but process p sleep at line 
126 with b rn E = true {b m E is the local variable b of procedure moveElement) . 
Process p continues the migration and updates the new current index when the 
migration completes. Then, process p does several insertions to let the occ of 
the current hash table reach one more than its bound. Just at that moment, 
P — 1 processes wake up, increase the occ of the current hash table to be P — 1 
more, and return to line 30. Since P — 1 processes insert different values in the 
hash table, after P — 1 processes finish their insertions, the occ of the current 
hash table reaches 2P — 1 more than its bound. 

It may be useful to make size larger than bound + IP to avoid too many 
collisions, e.g. with a constraint size > a ■ bound for some a > 1. If we did not 
introduce dels, every migration would force the sizes to grow, so that our hash 
table would require unbounded space for unbounded life time. We introduced 
dels to avoid this. 

Strictly speaking, instruction 82 inspects one shared variable, H [index], and 
modifies three other shared variables, viz. H[i], Heap[H_ index], and H_ index. 
In general, we split such multiple shared variable accesses in separate atomic 
commands. Here the accumulation is harmless, since the only possible interfer- 
ences are with other allocations at line 82 and deallocations at line 71. In view 
of the invariant Ha2, all deallocations are at pointers h < H_ index. Allocations 
do not interfere because they contain the increment H_index++ (see procedure 
allocate) . 

The procedure newTable first searches for a free index i, say by round robin. 
We use a nondeterministic choice. Once a free index has been found, a hash table 
is allocated and the index gets an indirection to the allocated address. Then 
the current index gets a next pointer to the new index, unless this pointer has 
been set already. 

The variables prot[i] are used primarily as counters with atomic increments 
and decrements. In 78, however, we use an atomic test-and-set instruction. 
Indeed, separation of this instruction in two atomic instructions is incorrect, 
since that would allow two processes to grab the same index i concurrently. 

3.5.5 Procedure migrate 

After the choice of the new hash table, the procedure migrate serves to transfer 
the contents in the current hash table to the new hash table by calling a pro- 
cedure moveContents and to update the current hash table pointer afterwards. 
Migration is complete when at least one of the (parallel) calls to migrate has 
terminated. 



94: 
95: 
97: 
98: 



proc migrate{) = 

local i : . . 2P; h : pointer to Hashtable ; b : Bool ; 
i := next [index]; 
prot[i]++ ; 

if index ^ currlnd then 



prot[i]~ ; 
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else 

99: busy[z]++ ; 

100: h := H[i] ; 

101: if index = currlnd then 

moveContents(E[index],h) ; 
103: ( b := (currlnd = index) ; 

if b then currlnd :— i ; {Y := H[i]. table } 
end ) ; 
if 6 then 
104: busy[indcx] — ; 

105: prot[index] — ; 

end end ; 
release Access (i) ; 
end end . 

According to invariants mi4 and mi5, it is an invariant that i = next (index) ^ 
holds after instruction 94. 

Line 103 contains a compare&swap instruction to update the current hash ta- 
ble pointer when some process finds that the migration is finished while currlnd 
is still identical to its index, which means that i is still used for the next current 
hash table (see invariant mi,5). The increments of prot[z] and busy[i] here are 
needed to protect the next hash table. The decrements serve to avoid memory 
loss. 

3.5.6 Procedure refresh 

In order to avoid that a delayed process starts migration of an old hash table, 
we encapsulate migrate in refresh in the following way. 

proc refresh() = 
90: if index =^ currlnd then 

releaseAccess(index) ; 
getAccess() ; 
else migrateQ end ; 
end. 

When index is outdated, the process needs to call releaseAccess to abandon 
its hash table and getAccess to acquire the present pointer to the current hash 
table. Otherwise, the process can join the migration. 

3.5.7 Procedure moveContents 

Procedure moveContents has to move the contents of the current table to the 
next current table. All processes that have access to the table, may also partici- 
pate in this migration. Indeed, they cannot yet use the new table (see invariants 
JVel and Ne3). We have to take care that delayed actions on the current table 
and the new table are carried out or abandoned correctly (see invariants Cul 
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and mElO). Migration requires that every value in the current table be moved 
to a unique position in the new table (see invariant Nel9). 

Procedure moveContents uses a private variable toBeMoved that ranges over 
sets of locations. The procedure is given by 

proc moveContents(from, to : pointer to Hashtable) = 

local i : Nat ; b : Bool ; v : EValue ; toBeMoved : set of Nat ; 
toBeMoved := {0, . . . , from.size — 1} ; 
while cur rind = index A toBeMoved ^ do 
choose i € toBeMoved ; 
v := from.tablefi] ; 
if v = done then 

toBeMoved := toBeMoved - {i} ; 
else 

{ b := (v = from. table[z]) ; 

if b then from. table[i] := old(val(v)) end } ; 
if b then 

if vaJ(u) 7^ null then moveElement(val(v) , to) end; 
from. t able [i] := done ; 
toBeMoved := toBeMoved - {i} ; 
end end end ; 
end . 

Note that the value is tagged as outdated before it is copied (see invariant 
mCll). After tagging, the value cannot be deleted or assigned until the migra- 
tion has been completed. Tagging must be done atomically, since otherwise an 
interleaving deletion may be lost. When indeed the value has been copied to the 
new hash table, it becomes done in the old hash table in line 117. This has the 
effect that other processes need not wait for this process to complete procedure 
moveElement, but can help with the migration of this value if needed. 

Since the address is lost after being rewritten to done, we had to introduce 
the shared auxiliary hash table Y to remember its value for the proof of correct- 
ness. This could have been avoided by introducing a second tagging bit, say for 
"very old" . 

The processes involved in the same migration should not use the same strat- 
egy for choosing i in line 111, since it is advantageous that moveElement is 
called often with different values. They may exchange information: any of 
them may replace its set toBeMoved by the intersection of that set with the set 
toBeMoved of another one. We do not give a preferred strategy here, one can 
refer to algorithms for the write-all problem [71 HH], 

3.5.8 Procedure moveElement 

The procedure moveElement moves a value to the new hash table. Note that 
the value is tagged as outdated in moveContents before moveElement is called. 

proc moveElement(v : Value \ {null}, to : pointer to Hashtable) = 



110: 
111: 



112: 
114: 



116: 
117: 
118: 
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local a : Address ; k,m,n : Nat ; w : EValue ; b : Bool ; 
120: rt := ; & := false ; a := ADK(d) ; m := to. size ; 

repeat 

121: k := Jtey(a, m, n) ; w := to.table[/c] ; 

if w — null then 
123: ( b := (to.table[fc] = null); 

if b then to.table[fe] := v end } ; 
else n++ end ; 
125: until b V a = ADft(iu) V currlnd ^ index ; 

126: if b then to.occ++ end 

end . 

The value is only allowed to be inserted once in the new hash table (see 
invariant Nel9), since otherwise the main property of open addressing would 
be violated. In total, four situations can occur in the procedure moveElement: 

• the current location k contains a value with a different address. The 
process increases n to inspect the next location. 

• the current location k contains a value with the same address. This means 
that the value has already been copied to the new hash table, the process 
therefore terminates. 

• the current location k is an empty slot. The process inserts v and returns. 
If insertion fails, since another process filled the empty slot in between, 
the search is continued. 

• when index happens to differ from currlnd, the entire migration has been 
completed. 

While the current hash table pointer is not updated yet, there exists at least 
one null entry in the new hash table (see invariants JVeS, Ne22 and Ne23), 
hence the local variable n in the procedure moveElement never goes beyond the 
size of the hash table (see invariants mE3 and mE8), and the termination is 
thus guaranteed. 

4 Correctness (Safety) 

In this section, we describe the proof of safety of the algorithm. The main 
aspects of safety are functional correctness, atomicity, and absence of memory 
loss. These aspects are formalized in eight invariants described in section ITTl 
To prove these invariants, we need many other invariants. These are listed in 
Appendix A. In section fOl we sketch the verification of some of the invariants by 
informal means. In section fOl we describe how the theorem prover PVS is used 
in the verification. As exemplified in 14.21 Appendix B gives the dependencies 
between the invariants. 

Notational Conventions. Recall that there are at most P processes with 
process identifiers ranging from 1 up to P. We use p, q, r to range over process 
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identifiers, with a preference for p. Since the same program is executed by all 
processes, every private variable name of a process ^ p is extended with the 
suffix "." + "process identifier". We do not do this for process p. So, e.g., the 
value of a private variable x of process q is denoted by x.q, but the value of x 
of process p is just denoted by x. In particular, pc.q is the program location of 
process q. It ranges over all integer labels used in the implementation. 

When local variables in different procedures have the same names, we add 
an abbreviation of the procedure name as a subscript to the name. We use the 
following abbreviations: fi for find, del for delete, ins for insert, ass for assign, 
gA for getAccess, rA for release Access, nT for newTable, mig for migrate, ref 
for refresh, mC for moveContents, mE for moveElement. 

In the implementation, there are several places where the same procedure is 
called, say getAccess, release Access, etc. We introduce auxiliary private vari- 
ables return, local to such a procedure, to hold the return location. We add a 
procedure subscript to distinguish these variables according to the above con- 
vention. 

If V is a set, |V denotes the number of elements of V. If b is a boolean, 
then jj& = Q when b is false, and $6 = 1 when b is true. Unless explicitly defined 
otherwise, we always (implicitly) universally quantify over addresses a, values 
v, non-negative integer numbers fc, m, and n, natural number I, processes p, q 
and r. Indices i and j range over [1, 2P], We abbreviate H(currlnd).size as 
cur Size. 

In order to avoid using too many parentheses, we use the usual binding order 
for the operators. We give "A" higher priority than "V". We use parentheses 
whenever necessary. 

4.1 Main properties 

We have proved the following three safety properties of the algorithm. Firstly, 
the access procedures End, delete, insert, assign, are functionally correct. Sec- 
ondly they are executed atomically. The third safety property is absence of 
memory loss. 

Functional correctness of find, delete, insert is the condition that the result 
of the implementation is the same as the result of the specification (fS), (dS), 
(iS). This is expressed by the required invariants: 

Col: pc — 14 =>■ val(r fi ) = rSfi 

Co2: pc G {25, 26} suc de i = sucS d ei 

Co3: pc <E {41,42} =>■ suci„ s = sucSi ns 

Note that functional correctness of assign holds trivially since it does not 
return a result. 

According to the definition of atomicity in chapter 13 of [23, atomicity means 
that each execution of one of the access procedures contains precisely one execu- 
tion of the corresponding specifying action (fS), (dS), (iS), (aS). We introduced 
the private auxiliary variables cnt to count the number of times the specifying 
action is executed. Therefore, atomicity is expressed by the invariants: 
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Cnl. 
Cn2: 
Cn3. 
Cn4. 



pc = 14 cntfi = 1 
pc £ {25,26} => cnt dei = 1 
pc 6 {41,42} => cnt ms = 1 
pc — 57 =>■ cnt ass = 1 



We interpret absence of memory loss to mean that the number of allocated 
hash tables is bounded. More precisely, we prove that this number is bounded 
by 2P. This is formalized in the invariant: 

Nol: tt{fc | k < H_index A Heap(fc) ^ _L} < 2P 

An important safety property is that no process accesses deallocated mem- 
ory. Since most procedures perform memory accesses, by means of pointers that 
are local variables, the proof of this is based on a number of different invariants. 
Although, this is not explicit in the specification, it has been checked because 
the theorem prover PVS does not allow access to deallocated memory as this 
would violate type correctness conditions. 

4.2 Intuitive proof 

The eight correctness properties (invariants) mentioned above have been com- 
pletely proved with the interactive proof checker of PVS. The use of PVS did 
not only take care of the delicate bookkeeping involved in the proof, it could 
also deal with many trivial cases automatically. At several occasions where PVS 
refused to let a proof be finished, we actually found a mistake and had to correct 
previous versions of this algorithm. 

In order to give some feeling for the proof, we describe some proofs. For the 
complete mechanical proof, we refer the reader to [H]. Note that, for simplic- 
ity, we assume that all non-specific private variables in the proposed assertions 
belong to the general process p, and general process q is an active process that 
tries to threaten some assertion (p may equal q). 

Proof of invariant Col (as claimed in I4.1f> . According to Appendix B, the 
stability of Col follows from the invariants Ot3, 61, RIO, which are given in 
Appendix A. Indeed, Ot3 implies that no procedure returns to location 14. 
Therefore all return statements falsify the antecedent of Col and thus preserve 
Col . Since r-p and rSfi are private variables to process p, Col can only be 
violated by process p itself (establishing pc at 14) when p executes 13 with 
rji = null V afi — ADR(rfi). This condition is abbreviated as Find(rfi,a,fi). 
Invariant G10 then implies that action 13 has the precondition val{rfi) = rSfi, 
so then it does not violate Col. In PVS, we used a slightly different definition 
of Find, and we applied invariant hi to exclude that rfi is done or del, though 
invariant El is superfluous in this intuitive proof. □ 

Proof of invariant Ot3. Since the procedures getAccess, release Access, refresh, 
newTable are called only at specific locations in the algorithm, it is easy to list 
the potential return addresses. Since the variables return are private to process 
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p, they are not modified by other processes. Stability of Ot3 follows from this. 
As we saw in the previous proof, Ot3 is used to guarantee that no unexpected 
jumps occur. □ 

Proof of invariant 610. According to Appendix B, we only need to use 69 
and Ot3. Let us use the abbreviation k = key(a,fi,lfi,nfi). Since rp and rSfi 
are both private variables, they can only be modified by process p when p is 
executing statement 7. We split this situation into two cases 

1. with precondition Find(hfi. table[fc], a-p,) 

After execution of statement 7, rfi becomes hfi.ta.ble[k], and rSfi becomes 
X(ay;). By 69, we get val(rfi) = rSfi. Therefore the validity of 610 is 
preserved. 

2. otherwise. 

After execution of statement 7, rfi becomes hfi. table [k], which then fal- 
sifies the antecedent of BIO. □ 



Proof of invariant 69. According to Appendix B, we proved that 69 follows 
from Ax2, 61, 63, 64, 65, 68, Ha4, Hc4, Cul, Cu9, CulO, and Cull. We 
abbreviate key(afi,lfi,rifi) as k. We deduce hfi — H(index) from 64, E(index) is 
not _L from He4, and k is below H(index).size from Ax2, 64 and 63. Wc split 
the proof into two cases: 

1. index ^ currlnd: By Ha4, it follows that H(index) ^ H(currlnd). Hence 
from Cul, we obtain hfi.table[k] = done, which falsifies the antecedent 
of 69. 

2. index = currlnd: By premise Find(hfi.ta.ble[k], dfi), we know that hfi. table 
done because of 61. By Cu9 and CulO, we obtain va7(/i/i.table[£;]) = 
val(Y[k]). Hence it follows that Find(Y[k],a,fi). Using 68, we obtain 

Vra < nfi : ^Find(Y[kcy(afi, curSize, m)],afi) 

We get nfi is below curSize because of 65. By Cull, we conclude 

X(afi) — val(hfi. table [k]) 

□ 



4.3 The model in PVS 

Our proof architecture (for one property) can be described as a dynamically 
growing tree in which each node is associated with an assertion. We start 
from a tree containing only one node, the proof goal, which characterizes some 
property of the system. We expand the tree by adding some new children 
via proper analysis of an unproved node (top-down approach, which requires a 
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good understanding of the system). The validity of that unproved node is then 
reduced to the validity of its children and the validity of some less or equally 
deep nodes. 

Normally simple properties of the system are proved with appropriate prece- 
dence, and then used to help establish more complex ones. It is not a bad thing 
that some property that was taken for granted turns out to be not valid. Indeed, 
it may uncover a defect of the algorithm, but in any case it leads to new insights 
in it. 

We model the algorithm as a transition system [2] , which is described in the 
language of PVS in the following way. As usual in PVS, states are represented 
by a record with a number of fields: 

State : TYPE = [# 
% global variables 

busy : [ range(2*P) — > nat ], 
prot : [ range(2*P) — * nat ], 

% private variables: 

index : [ range(P) — > range(2*P) ], 

pc : [ range(P) — > nat ], % private program counters 

% local variables of procedures, also private to each process: 
% find 

a_find : [ range(P) — > Address ], 
r_find : [ range (P) — » E Value ], 

% getAccess 

return- get Access : [ range(P) — > nat ], 

#]"' 

where range(P) stands for the range of integers from 1 to P. 

Note that private variables are given with as argument a process identifier. 
Local variables are distinguished by adding their procedure's names as suffixes. 

An action is a binary relation on states: it relates the state prior to the action 
to the state following the action. The system performed by a particular process 
is then specified by defining the precondition of each action as a predicate on 
the state and also the effect of each action in terms of a state transition. For 
example, line 5 of the algorithm is described in PVS as follows: 

% corresponding to statement find5: h := H [index]; n :— 0; 
find5(i,sl,s2) : bool = 

pc(sl)(i)=5 AND 

s2 = si WITH [ (pc)(i) := 6, 

(n_find)(i) := 0, 

(h_find)(i) := H(sl)(index(sl)(i)) ] 



where i is a process identifier, si is a pre-state, s2 is a post-state. 
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Since our algorithm is concurrent, the global transition relation is defined as 
the disjunction of all atomic actions. 

% transition steps 
step(i,sl,s2) : bool = 

find5(i,sl,s2) or find6(i,sl,s2) or ... 

deletel5(i,sl,s2) or deletel6(i,sl,s2) or ... 

Stability for each invariant is proved by a PVS Theorem of the form: 

% Theorem about the stability of invariant filO 
IV-filO: THEOREM 

forall (u,v : state, q : range(P) ) : 

step(q,u,v) AND filO(u) AND fi9(u) AND ot3(u) 

=> filO(v) 

To ensure that all proposed invariants are stable, there is a global invariant INV, 
which is the conjunction of all proposed invariants. 

% global invariant 
INV(s:state) : bool = 

He3(s) and He4(s) and Cul(s) and ... 



% Theorem about the stability of the global invariant INV 
IV-INV: THEOREM 

forall (u,v : state, q : range(P) ) : 

step(q,u,v) AND INV(u) => INV(v) 

We dehne Init as all possible initial states, for which all invariants must be 
valid. 

% initial state 
Init: { s : state [ 

(forall (p: range(P)): 
pc(s)(p)=0 and ... 
...) and 
(forall (a: Address): 
X(s)(a)=null) and 

} 

% The initial condition can be satisfied by the global invariant INV 
IV- Init: THEOREM 
INV(Init) 

The PVS code contains 11 preconditions to imply well-dcfincdness: e.g. in Bnd7, 
the hash table must be non-NIL and I must be its size. 



% corresponding to statement find7 
find7(i,sl,s2) : bool = 

i?(Heap(sl)(h_find(sl)(i))) and 
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l_find(sl)(i)=size(i_(Heap(sl)(h_find(sl)(i)))) and 
pc(sl)(i)=7 and 

All preconditions are allowed, since we can prove lock-freedom in the fol- 
lowing form. In every state si that satisfies the global invariant, every pro- 
cess q can perform a step, i.e., there is a state s2 with (sl,s2) s step and 
pc(sl, q) 7^ pc(s2, q). This is expressed in PVS by 

% theorem for lock-freedom 
IV_prog: THEOREM 

forall (u: state, q: range(P) ) : 

INV(u) => exists (v: state): pc(u)(q) /= pc(v)(q) and step(q,u,v) 

5 Correctness (Progress) 

In this section, we prove that our algorithm is lock-free, and that it is wait-free 
for several subtasks. Recall that an algorithm is called lock-free if always at 
least some process will finish its task in a finite number of steps, regardless of 
delays or failures by other processes. This means that no process can block the 
applications of further operations to the data structure, although any particular 
operation need not terminate since a slow process can be passed infinitely often 
by faster processes. We say that an operation is wait-free if any process involved 
in that operation is guaranteed to complete it in a finite number of its own steps, 
regardless of the (in)activity of other processes. 

5.1 The easy part of progress 

It is clear that releaseAccess is wait-free. It follows that the wait-freedom of 
migrate depends on wait-freedom of moveContents. The loop of moveContents 
is clearly bounded. So, wait-freedom of moveContents depends on wait-freedom 
of moveElcment. It has been proved that n is bounded by m in moveElement 
(see invariants mE3 and mE8). Since, moreover, to.table[fc] ^ null is sta- 
ble, the loop of moveElcment is also bounded. This concludes the sketch that 
migrate is wait-free. 

5.2 Progress of newTable 

The main part of procedure newTable is wait-free. This can be shown informally, 
as follows. Since we can prove the condition next (index) ^ is stable while 
process p stays in the region [77, 84], once the condition next (index) ^ holds, 
process p will exit newTable in a few rounds. 

Otherwise, we may assume that p has precondition next (index) = before 
executing line 78. By the invariant 

Ne5: pee [1,58] V pc > 62 A pc ^ 65 A next (index) = 
=>■ index = currlnd 
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we get that index = currlnd holds and next(currlnd) = from the precondi- 
tion. We define two sets of integers: 



and consider the sum ^ i _ 1 (|t(p?'iS , etl(i)) + ${P r Set2(i))). While process p 
is at line 78, the sum cannot exceed 2P — 1 because there are only P pro- 
cesses around and process p contributes only once to the sum. It then fol- 
lows from the pigeon hole principle that there exists j G [1,2P] such that 
§(prSetl(j)) + §(prSet2(j)) = and j ^ index.p. By the invariant 



we can get that prot[j] = because of j ^ index.p = currlnd. 

While currlnd is constant, no process can modify prot[j] for j ^ currlnd 
infinitely often. Therefore, if process p acts infinitely often and chooses its value 
i in 78 by round robin, process p exits the loop of ncwTable eventually. This 
shows that the main part of newTable is wait-free. 

5.3 The failure of wait-freedom 

Procedure getAccess is not wait-free. When the active clients keep changing 
the current index faster than the new client can observe it, the accessing client 
is doomed to starvation. In that case, however, the other processes repeatedly 
succeed. It follows that getAccess, refresh, and newTable are lock-free. 

It may be possible to make a queue for the accessing clients which is emptied 
by a process in newTabfe. The accessing clients must however also be able to 
enter autonomously. This would at least add another layer of complications. 
We therefore prefer to treat this failure of wait-freedom as a performance issue 
that can be dealt with in practice by tuning the sizes of the hash tables. 

According to the invariants B5, de8, in8 and as6, the primary procedures 
Gnd, delete, insert, assign are loops bounded by n < h.slze, and n is only reset 
to during migration. If n is not reset to 0, it is incremented or stays constant. 
Indeed, the atomic if statements in 18b, 35b, and 50b have no else parts. In 
delete and assign, it is therefore possible that n stays constant without termi- 
nation of the loop. Since assign can modify non-null elements of the table, it 
follows that delete and assign are not wait-free. This unbounded fruitless activ- 
ity is possible only when assign actions of other processes repeatedly succeed. It 
follows that the primary procedures are lock-free. This concludes the argument 
that the system is lock-free. 



prSetl (i) 
prSet2(i) 



{r | indcx.r = i A per £ {0, 59, 60}} 
{r | indcx.r = i A per G {104, 105} 

V i r A-r = i A indcx.r ^ i A per G [67, 72] 

V i nT .r = i A per G [81, 84] 

V imig-r = i A per > 97 } 



prl: 



protbl = UprSctl(j)) + $(prSet2(j)) + ft(currlnd = j) 
+f|(next(currlnd) = j) 
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6 Conclusions 

Lock-free shared data objects are inherently resilient to halting failures and 
permit maximum parallelism. We have presented a new practical, lock-free al- 
gorithm for concurrently accessible hash tables, which promises more robust per- 
formance and reliability than a conventional lock-based implementation. More- 
over, the new algorithm is dynamic in the sense that it allows the hash table to 
grow and shrink as needed. 

The algorithm scales up linearly with the number of processes, provided the 
function key and the selection of i in line 111 are defined well. This is confirmed 
by some experiments where random values were stored, retrieved and deleted 
from the hash table. These experiments indicated that 10 6 insertions, deletions 
and finds per second and per processor are possible on an SGI powerchallenge 
with 250Mhz R12000 processors. This figure should only be taken as a rough 
indicator, since the performance of parallel processing is very much influenced 
by the machine architecture, the relative sizes of data structures compared to 
sizes of caches, and even the scheduling of processes on processors. 

The correctness proof for our algorithm is noteworthy because of the extreme 
effort it took to finish it. Formal deduction by human-guided theorem proving 
can, in principle, verify any correct design, but doing so may require unrea- 
sonable amounts of effort, time, or skill. Though PVS provided great help for 
managing and reusing the proofs, we have to admit that the verification for our 
algorithm was very complicated due to the complexity of our algorithm. The 
total verification effort can roughly be estimated to consist of two man year ex- 
cluding the effort in determining the algorithm and writing the documentation. 
The whole proof contains around 200 invariants. It takes an lGhz Pentium IV 
computer around two days to re-run an individual proof for one of the biggest 
invariants. Without suitable tool support like PVS, we even doubt if it would 
be possible to complete a reliable proof of such size and complexity. 

It may well be possible to simplify the proof and reduce the number of in- 
variants slightly, but we did not work on this. The complete version of the PVS 
specifications and the whole proof scripts can be found at 0. Note that we 
simplified some definitions in the paper for the sake of presentation. 



A Invariants 

We present here all invariants whose validity has been verified by the theorem 
prover PVS. 

Conventions. We abbreviate 

Find(r, a) = r = null V a = ADR(r) 

LeastFind(a,n) = (Vm < n : -^Find(Y[key(a, curSize,m)], a)) 

A Find(Y[key(a, curSize, n)],a)) 
LeastFind(h,a,n) = (Vm < n : -^Find(h.ta.ble[kcy(a, ft,. size, m)], a)) 
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A Find(h.table[key(a, h.size, n)],a)) 
Axioms on functions key and ADR 



Axl 
Ax2. 
Ax3 



v = null = ADR(v) = 
< key(a,l,k) < I 

0<k<m<l=> key(a, I, k) ^ key(a, I, m) 



Main correctness properties 



Col 

Co2 

Co3 

Cnl 

Cn2. 

Cn3 

Cn4 



pc = 14 => val(rfi) = rSfi 

pc e {25, 26} suc de i = SUcSdel 

pc e {41,42} => SUC ins = SUcS tns 

pc = 14 => cntfi = 1 
pee {25, 26}^ cnt de i = 1 
pc e {41,42} cnt ms = 1 
pc = 57 => cnt ass 



1 



The absence of memory loss is shown by 

Nol: $(nbSetl) <2*P 
No2: {(nbSetl) = ${nbSet2) 

where nbSetl and nbSet2 are sets of integers, characterized by 

nbSetl = {k\k< H_ index A Heap(fc) ^ _L} 
nbSet2 = {i \ H(i) ^ V (3r : per = 71 A i rA .r = i)} 

Further, we have the following definitions of sets of integers: 



25 A suc del .r} 



return g A-r > 30 
.r > 30) 



deSetl = {k\k < curSizc A Y[k] = del} 
deSet2 = {r \ indcx.r = currlnd A per ■ 
deSet3 = {k \ k < H(next(currlnd)).size 

A H(next(currlnd)).table[fc] = del} 
ocSetl = {r | indcx.r ^ currlnd 

V per e [30, 41] V per G [46, 57] 

V per E [59, 65] A return g A-r > 30 

V per £ [67, 72] A (return r A-r = 59 A 

V return r A-r — 90 A return re f. 

V (per = 90 V per G [104,105]) A return ref .r > 30} 
ocSet2 = {r \ per > 125 A b mE .r A to.r = H(currlnd)} 
ocSet3 = {r \ indcx.r = currlnd A per = 41 A suc lns .r 

V indcx.r = currlnd A per = 57 A r ass .r = null} 
ocSet4 = {k\k < curSize A vaJ(Y[fc]) ^ null} 

ocSet5 = {k | k < H(next(currlnd)).size 

A vaJ(H(next(currInd)).table[fc]) ^ null} 

ocSet6 = {k | k < H(next(currlnd)).size 

A H(next(currlnd)).table[fc] ^ null} 

ocSet7 = {r \ per > 125 A b m E.r A to.r = H(next(currlnd))} 
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prSctl(i) = {r | indcx.r = i A per £ {0,59,60}} 
prSet2(i) = {r \ index.r = i A per G {104, 105} 

V i r A-r = i A indcx.r ^ i A per G [67, 72] 

V i nT .r = i A per G [81,84] 

V i mig .r = i A per > 97} 

prSet3(i) = {r \ indcx.r = i A per G [61, 65] U [104, 105] 

V i r A-i~ = i A per = 72 

V i nT .r = i A per e [81,82] 

V i mig .r = i A per G [97,98]} 
prSet4(i) — {r \ indcx.r — i A per G [61,65] 

V i mi9 .r = i A per G [97,98]} 
buSctl (i) = {r | index.r = i 

A (per G [1, 58] U (62, 68] A per ^ 65 

V per G [69, 72] A return rA .r > 59 

V per > 72)} 

buSct2(i) = {r \ index.r = i A per = 104 

V v^.r = i A indcx.r ^ i A per G [67, 68] 

V i„ T .r = i A per G [82,84] 

V imig.r — i A per > 100} 

We have the following invariants concerning the Heap 

Hcl: Heap(0) = _L 

He2: R(i) ^ = Heap(H(i)) ^ _L 

Jfe3: Heap(H(currInd)) ^ _L 

He4: pc G [1, 58] V pc> 65 A -.(pc G [67, 72] A i r A = index) 

Heap(H(indcx)) ^ ± 

He5: Heap(H(«)) ^ _L H(i).size > P 

Hc6: next(currlnd) ^ =>• Heap(H(next(currInd))) ^ ± 

Invariants concerning hash table pointers 

Hal: H_index>0 

Ha2: R(i) < H_ index 

Ha3: i^j A Heap(H(i)) ^ _L => H(i) ^ H(j) 

Ha4: index ^ currlnd H(indcx) 7^ H(currlnd) 

Invariants about counters for calling the specification. 

Cn5: pee [6,7] cniyi = 
Cn6: pc G [8, 13] 

V pc G [59, 65] A return gA = 10 

V pc G [67, 72] A (return rA = 59 A return gA = 10 

V return r A — 90 A return re f = 10 

V pc > 90 A return re f = 10 

=4> cn^ = (t(ryi = null V ay; = ADR(r fi )) 
Cn7: pc G [16, 21] A pc ^ 18 
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V pc e [59, 65] A return gA = 20 

V pc e [67, 72] A (return rA = 59 A return gA = 20 

V return r A — 90 A return re f = 20 

V pc > 90 A return re f = 20 
=> cni de i = 

Cn8: pc = 18 => cni de ; = jj(r de i = null) 
Cn9: pc G [28, 33] 

V pc G [59, 65] A return gA = 30 

V pc G [67, 72] A (return rA = 59 A return gA = 30 

V return r A = 77 A return„T = 30 

V return r A — 90 A return re f = 30 

V pc G [77, 84] A return nT = 30 

V pc > 90 A returnref = 30 
=> cni ms = 

CnIO: pc G [35,37] 

V pc G [59, 65] A return g A — 36 

V pc G [67, 72] A (return rA — 59 A return gA = 36 

V return rA — 90 A return re f — 36 

V pc > 90 A return re f = 36 

=4- cni ins = jj(a ms = AM(r ms ) V suc^) 
CniJ: pc G [44,52] 

V pc G [59, 65] A returngA G {46, 51} 

V pc G [67, 72] A (return rA = 59 A return gA G {46, 51} 

V return r A = 77 A return n T = 46 

V return r A — 90 A return re f G {46, 51} 

V pc G [77, 84] A return nT = 46 

V pc > 90 A returnref G {46, 51} 
=> cnt ass sign = 

Invariants about old hash tables, current hash table and the auxiliary hash 
table Y. Here, we universally quantify over all non-negative integers n < curSize. 

Cul: H(index) 7^ H(currlnd) A k < E(index). size 

A (pc G [1, 58] V pc> 65 A ^(pc G [67, 72] A i rA = index) 
=> H(indcx).table[fc] = done 

Cu2: I k < curSize A Y[fc] ^ null}) < curSize 

Cu3: H(currlnd). bound +2* P < curSize 

Cu4: H(currlnd).dels + $(deSet2) = %(deSetl) 

Cu5: Cu5 has been eliminated. The numbering has been kept, so as not to 

endanger the consistency with Appendix B and the PVS script. 
Cu6: H(currInd).occ + $(ocSetl) + ${ocSet2) < H(currInd).bound + 2 * P 
Cu7: t)({fc I k < curSize A Y[fc] ^ null} 

= H(currInd).occ + $(ocSet2) + i(ocSet3) 
Cu8: next(currlnd) = => -> oJdp(H(currInd).table[n]) 
Cu9: -n(oldp(H(currInd).table[n])) H(currlnd). table [n] = Y[n] 
CulO: o7dp(H(currInd). table [n]) A vai(H(currInd).table[n]) ^ null 
vaJ(H(currInd).table[n]) = vaJ(Y[n]) 
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Cull: LeastFind(a,n) => X(a) = val(Y[key(a, curSize, n)]) 
Cul2: X(a) = va/(Y [.key (a, curSize, n)]) 7^ null LeastFind(a,n) 
Cul3: X(a) = vaJ(Y[key(a, curSize, n)]) 7^ null A n^ra < curSize 

=4> AGR(Y[k:cy(a, curSize, ra)]) 7^ a 
CuI4: X(a) = null A va](Y[key(a, curSize, n)j) 7^ null 

=> ADR(Y[Jfcj(a, curSize, n)\) ^ a 
Cul5: X(a) 7^ null 

=> 3m < curSize : X(a) = vai(Y[iey(a, curSize, m)\) 
Cul6: 3(f : [{m : < m < curSize) A vaJ(Y[m]) 7^ null} -> 

{u : u 7^ null A (3fc < curSize : v = vai(Y[fc]))}]) : 

/ is bijective 

Invariants about next and next(currlnd): 
Nel: currlnd 7^ next(currlnd) 

Ne2: next(currlnd) 7^ => next(next(currlnd)) = 
Ne3: pc G [1, 59] V pc > 62 A pc 7^ 65 index 7^ next(currlnd) 
Ne4: pc G [1, 58] V pc > 62 A pc 7^ 65 => index ^ next(indcx) 
Ne5: pee [1, 58] V pc > 62 A pc 7^ 65 A next (index) = 

=> index = currlnd 
JVe6: next(currlnd) 7^ 

=> tt(ocSet6) < I fc < curSize A Y[fc] 7^ null} - H(currlnd).dels 
-$(deSet2) 
Nc7: next (currlnd) 7^ 

=> H(currlnd) .bound — H(currlnd).dels + 2 * P 
< H(next(currlnd)). bound 
IVe<5: next(currlnd) 7^ 

=> H(next(currlnd)). bound +2 * P < H(next(currlnd)).size 
Ne9: next(currlnd) 7^ => H(next(currlnd)).dels = tt(deSet3) 
Ne9a: next(currlnd) 7^ => H(next(currlnd)).dels = 
JVeiO: next(currlnd) 7^ A k < /i.size =>• /i.table[fc] {del, done}, 

where h = H(next(currlnd)) 
Nell: next(currlnd) 7^ A k < H(next(currlnd)).size 

=>■ -ioWp(H(next (currlnd) ) .table [k] ) 
Nel2: k < curSize A H(currlnd).table[/c] = done A m < /i.size 

A LcastFind(h, a, m) 

=> X(a) = vai(/i.table[key(a, /i.size, m)]), 
where a = ADR(Y[k]) and h — H(next(currlnd))) 
Nel3: k < curSize A H(currlnd).table[fc] = done A m < /i.size 
A X(a) = va_!(/i.table[Jcey(a, /i.size, m)]) 7^ null 
=>■ LcastFind(h,a,m), 

where a = ADK(Y[fc]) and /i = H(next(currlnd)) 
Nel4: next(currlnd) 7^ A a 7^ A k < /i.size 

A X(a) = vai(/i.table[key(a, /i.size, fc)]) 7^ null 
LcastFind(h, a, k), 

where h = H(next(currlnd)) 
Nel5: k < curSize A H(currlnd).table[fc] = done A X(a) 7^ null 
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A m < /i.size A X(a) = val(h.table[key(a, /i.size, m)]) 
A n < /i.size A m ^ n 
=> ADR(/i.table.[iej(a, /i.size, n)]) ^ a, 
where a = AGR(Y[fc]) and /i = H(next(currlnd)) 
JVel6: fc < curSizc A H(currlnd).table[/c] = done A X(a) = null 
A m < /i.size 

=>■ yaJ(/i.table[Jcej(a, /i.size, m)]) = null 

V ADR(h.table[key(a, /i.size, m)]) ^ a, 
where a = ADR(Y[fc]) and /i = H(next(currlnd)) 
Ncl7: next(currlnd) ^ A m < /i.size A a = AGR(/i.table[m]) ^ 
X(a) = vai(/i.table[m]) ^ null, 
where /i = H(next(currlnd)) 
Nel8: next(currlnd) / A m< /i.size A a = ADR(/i.table[m]) 7^ 
=> 3n < curSizc : vai(Y[n]) = vai(/i.table[m]) 

A oJdp(H(currInd).table[n]), 
where h = H(next(currlnd)) 
Ncl9: next(currlnd) / A m< /i.size A m ^ n < /i.size 
A a — ADR(h.table[key(a, /i.size, m)]) ^ 
=>- ADR(/i.table[Jcey(a, /i.size, n)]) 7^ a, 
where /i = H(next(currlnd)) 
Ne20: k < curSizc A H(currlnd). table [k] = done A X(a) ^ null 
=>■ 3m < /i.size : X(a) = va](/i.table[iey(a, /i.size, m)]), 
where a = ADK(Y[fc]) and /i = H(next(currlnd)) 
Nc21: Nc21 has been eliminated. 
Ne22: next(currlnd) ^ 

|(ocSet6) = H(next(currInd)).occ + (KocSeiT) 
JVe23: next(currlnd) ^ 

=> H(next(currInd)).occ < H(next(currlnd)). bound 
Nc24: next(currlnd) ^ => |}(ocSet5) < jj(ocSet4) 
IVe25: next(currlnd) ^ 

3(/ : [{m : < m < /i.size A vaJ(/i.table[m]) ^ null} — > 
null A (3fc < /i.size : u = vai(/i.table[fc]))}]) : 
/ is bijective, 
where h = H(next(currlnd)) 
Ne26: next(currlnd) ^ 

=> 3(f : [{« : 1; / null A (3m < /i.size : v — vai(/i.table[m]))} — > 
/ null A (3k :< curSize : v = vai(Y[fc]))}]) : 
/ is injective, 
where h = H(next(currlnd)) 
Ne27: next(currlnd) ^ A (3n < /i.size : vai(/i.table[n]) ^ null) 
=> 3(f : [{m : < m < /i.size A vai(/i.table[m]) ^ null} — > 
{fc : < fc < curSize A val(Y[k}) ^ null}]) 
/ is injective, 
where h = H(next(currlnd)) 

Invariants concerning procedure find (5. . . 14) 
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61: a fi ^0 

62: pc G {6, 11} => n fi =0 

63: pc G {7, 8, 13} => l fi = h fi .size 

64: pc G [6, 13] A pc ^ 10 =>• h fi = E(indcx) 

65: pc = 7 A hfi = H(currlnd) =4> n fi < curSize 

66: pc~8 A hfi — H(currlnd) A ^Find(rfi, cifi) A ryi ^ done 

=4- -i Find(Y[key(afi, curSize, nfi)], a fi ) 
67: pc — 13 A = H(currlnd) A -^Find(rfi, afi) A m < nfi 

=> -iFirid(Y[]fe7(ayj, curSize, m)],afi) 
68: pc G {7, 8} A /i^ = H(currlnd) A m < n fi 

=> -iFind(Y[]cey(a/;, curSize, m)}, afi) 
69: pc=7 A Find(t,a fi ) X(ay;) = val(i), 

where t = /i/;. table [Jcej(a^, Z/;, n^)] 
filO: pc^ (1,7] A Find{r fi , a fi ) va/fo) = rS*^ 
fill: pc = 8 A oldp(rfi) A index = currlnd 

=> next (currlnd) 7^ 

Invariants concerning procedure delete (15. . . 26) 
del: a del ^ 

de2: pc G {17, 18} =>■ Z de ; = h del .size 
de3: pc G [16, 25] A pc ^ 20 =>• ft dei = H(index) 
de4: pc = 18 ^ fc dei = key(a de i,ldel, n de i) 
de5: pc G {16, 17} V Deleting =$> ^suc de i 
de6: Deleting A sucS de i => r de i ^ null 

de7: pc = 18 A -1 oldp(h de i.table[k de i]) => h dei = H(currlnd) 
de8: pc G {17, 18} A h de i = H(currlnd) => n de i < curSize 
de9: pc = 18 A h de i = H(currlnd) 

A {val(r de i) ^ null V r del = del) 

=> r ^ null A (r = del V ADR(r) = ADR(r del )), 

where r = Y[key(a det , h dd .size, n de i)\ 
delO: pc G {17, 18} A h de i = H(currlnd) A m < n de i) 

-^Find(Y[key(a de i, curSize, m)],a de i) 
dell: pee {17,18} A Find(t,a de i) X(a del ) = val{t), 

where i = h de i.tab~Le[key{a de i,l de i,n del )] 
del2: pc = 18 A oldp(r de i) A index = currlnd 

=> next(currlnd) 7^ 
de!3: pc = 18 => k de i < R(indcx) .size 

where Deleting is characterized by 

Deleting = 

pc G [18, 21] V pc G [59, 65] A return gA = 20 

V pc G [67, 72] A (return rA = 59 A return gA = 20 

V return r A — 90 A return re f = 20) 

V pc > 90 A returnref — 20 
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Invariants concerning procedure insert (27. . . 52) 

inl: a ms = ADR(v ins ) A v ins ^ null 

in2: pc G [32,35] => Z ms = ft ins .size 

in3: pc G [28,41] A pc <£ {30,36} => /i ms = H(index) 

in4: pc G {33, 35} k ins = key(a ins ,l ms ,n ms ) 

in5: pc G [32, 33] V Inserting =>• ->suci ns 

in6: Inserting A sucS ins AD_R(r ins ) ^ a ms 

in7: pc = 35 A -i oldp(h ins .ta.ble[k ins ]) => h ins = H(currlnd) 

in8: pc G {33, 35} A h ins = H(currlnd) => n in;s < curSize 

in9: pc = 35 A /ij„ s = H(currlnd) 

A (va7(r ms ) ^ null V r ins = del) 

=> r ^ null A (r = del V ADR(r) = ADK(r ms )), 

where r = Y[Jcej(a ins , h ins .size, n ins )] 
inlO: pc G {32, 33, 35} A /i ins = H(currlnd) A m < n ins 

-^Find(Y[key(a ins , curSize, m)} , a ins ) 
mil: pc G {33, 35} A Find(t,ai ns ) => X(a,„ s ) = val(t), 

where t = h lns .table[kcy(a ins ,l ms 
in!2: pc = 35 A oldp(r lns ) A index = currlnd 

=>- next(currlnd) ^ 
inl3: pc = 35 =>• fc ms < H(index).size 



where Inserting is characterized by 
Inserting 

in g [35, 37] V pc G [59, 65] A return gA = 36 

V pc G [67, 72] A (return r A — 59 A return g A — 36 

V return r A — 90 A return re f = 36) 

V pc > 90 A return re f = 36 

Invariants concerning procedure assign (43. . . 57) 

asi: a ass = ADfi(« ass ) A w ass ^ null 

as2: pc G [48,50] =4- l ass = h ass .size 

as3: pc G [44, 57] A pc ^ {46, 51}^ h ass = H(index) 

as4: pc G {49, 50} k ass = key(a ass ,lass,n ass ) 

as5: pc = 50 A -i oJdp(/i as;s .table[fc ass ]) =>■ /i ass = H(currlnd) 

as6: pc = 50 A /i ass = H(currlnd) =>■ n ass < curSize 

as7: pc = 50 A /i ass = H(currlnd) 

A (val(r ass ) ^ null V r ass = del) 

r ^ null A (r = del V ADR(r) = ADR(r ass )), 

where r = Y[key(a ass ,h ass .size,n ass )] 
as8: pc G {48,49,50} A h ass = H(currlnd) A m < n ass 

=*> -.Fi:nd(Y[]tey(a ass , curSize, m)\ , a ass ) 
as9: pc = 50 A Find(t,a ass ) => X(a oss ) = val(t), 

where i = /i ass .table[.key(cw, l ass , n ass )] 
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aslO: pc = 50 A oldp(r ass sign) A index = currlnd 

=>■ next(currlnd) ^ 
asll: pc = 50 => k ass < E(indcx) .size 



Invariants concerning procedure releaseAccess (67. . . 72) 

rAl: h rA < H_ index 

rA2: pc G [70, 71] => h rA + 

rA3: pc = 71 => Heap(/i rj4 ) ^ _L 

rA4: pc = 71 => E(i rA ) = 

rA5: pc = 71 => /i rj4 7^ H(i) 

rA6: pc = 70 => E(i rA ) =t H(currlnd) 

rA7: pc = 70 

A (per G [1, 58] V per > 65 A -.(per G [67, 72] A i rj4 .r = index.r)) 

=> ti(i rA ) ^ ti(index.r) 
rA8: pc = 70 => i rA ^ next(currlnd) 
rA9: pc G [68, 72] A (h rA = V h rA ^ H(i rj4 )) 
H(va) = 

rA10: pc G [67, 72] A return rA G {0, 59} => i rj4 = index 
Mil: pc G [67,72] A return rA G {77,90} => t r i ^ index 
rA!2: pc G [67, 72] A return rA = 77 => next(index) ^ 
rA13: pc = 71 A per = 71 A p/r^> h rA ^ h rA .r 
rA14: pc = 71 A per = 71 A p 7^ r i rA ^i rA .r 

Invariants concerning procedure newTable (77. . . 84) 

E [81,82] => Heap(H(i nT )) = _L 
i [83,84]^ Heap(H(i„ T )) ^ ± 
= 84 => next(i„ T ) = 
i [83,84] => H(i nT ).dels = 
: [83,84]^ H(i„ T ).occ = 

i [83, 84] => H(i nT ).bound + 2 * P < H(i„ T ).size 
: [83,84] A index = currlnd 

H(currlnd). bound — H(currlnd).dels + 2 * P < H(i„r). bound 
i [83,84] A k < H(i nT ).size =>• H(i„ T ).table[fc] = null 
: [81,84] => i nT ^ currlnd 

: [81,84] A (per e [1,58] V per > 62 A per + 65) 
i„T 7^ index.r 

: [81,84] =>• i„ T 7^ next(currlnd) 
: [81,84] H(i„ T ) ^ H(currlnd) 

: [81,84] 

A (per G [1, 58] V per > 65 A -.(per G [67, 72] A i rA .r = index.r)) 

=4> H(i nT ) 7^ H(index.r) 
nT14: pc G [81,84] A per G [67,72] i nT ^ i rj4 .r 
nT15: pc G [83,84] A per G [67,72] => H(i nT ) ^ H(i rj4 .r) 
nT16: pc G [81,84] A per G [81,84] A p^r^ inT^inT-r 
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nT17: pee [81,84] A per G [95, 99] A indcx.r = currlnd 

=> InT ¥= irmg-r 

nT18: pee [81,84] A per > 99 => i nT ^ i mig .r 

Invariants concerning procedure migrate (94. . . 105) 

mil: pc = 98 V pc G {104, 105} =>■ index 7^ currlnd 

nn'2: pc > 95 =4> i m i S 7^ index 

mi3: pc = 94 ^> next (index) > 

nn'4: pc > 95 ^> i m j g 7^ 

mi5: pc > 95 => i m i g = next (index) 

mi6: per = 70 

A (pc G [95, 102) A index = currlnd V pc G [102, 103] V pc > 110) 

==> Z rj 4 .T ^ i mig 

mi7: pc £ [95, 97] A index = currlnd V pc > 99 

= ^ > i"rnig ^-©Xt(i m ^) 

miS: (pc G [95, 97] V pc G [99, 103] V pc > 110) A index = currlnd 

=> next(i ml9 ) = 
mi9: (pc G [95, 103] V pc > 110) A index = currlnd 

=> H(i ml9 ) 7^ H(currlnd) 
milO: (pc G [95, 103] V pc > 110) A index = currlnd 

A (per e [1, 58] V per > 62 A per ^ 65) 

=> H(i mis ) ^ H(index.r) 
mill: pc = 101 A index = currlnd V pc = 102 

^ hmig H(^mig) 

miJ2: pc > 95 A index = currlnd V pc G {102, 103} V pc > 110 

=> Heap(H(i mig )) ^ _L 
mil3: pc — 103 A index = currlnd A k < curSize 

=> H(index).table[fc] = done 
mil4: pc = 103 A index = currlnd A n < K(i m ig)- size 

A LeastFind(H(i mi9 ), a, n) 

=> X(o) = val(E(i mig )[key(a,H(i mig ).size,n)}) 
mil5: pc = 103 A index = currlnd A n < H(i rn i g ).s±ze 

A X(a) = vai(H(i mi5 ,).table[icey(a, H(i mig ).size, n)] 7^ null 

=> LeastFind(H(i mi9 ), a, n) 
nn'16: pc = 103 A index = currlnd A k < E(i mig ).size 

=> -ioidp(H(i mifl ). table [fc]) 
mil 7: pc = 103 A index = currlnd A X(a) 7^ null A k < ft,. size 

A X(a) = va/(/i.table[Jcey(a, ft. size, k)]) A kj^n < ft. size 
ADR(ft. table. [iey(a, ft. size, n)]) 7^0, 

where ft = n(i mig ) 
mil8: pc = 103 A index = currlnd A X(a) = null A k < ft. size 

=> val(h.table[key(a, h.size, k)]) — null 
V ADR(h.table[key(a, ft. size, k)]) 7^ a, 

where ft = K(i m ig) 
mil 9: pc = 103 A index = currlnd A X(a) 7^ null 
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=> 3m < ft. size : X(a) = val(h.table[key(a, ft,. size, to)], 
where ft = H(i mi9 ) 
mi20: pc = 117 A X(a) ^ null A va7(H(index).table[z mC ]) ^ null 
V pc > 126 A X(a) 7^ null A index = currlnd 
Vpc= 125 A X(a) 7^ null A index = currlnd 
A {b mE V val(w mE ) ^ null 
A a. mE = ADR(w mE )) 
=> 3m < ft. size : X(a) = val(h.ta.ble[key(a 1 ft. size, to)]), 
where a = ADR(Y[i m c]) and ft = H(next(currlnd)) 

Invariants concerning procedure movcContents (110. . . 118): 

mCl: pc = 103 V pc > 110 => to = H(i mi9 ) 
mC2: pc > 110 => from = H(index) 

mC3: pc > 102 A to e toBeMoved => m < H(indcx).size 

mC4: pc = 111 => 3m < from. size : to G toBeMoved 

mC5: pc > 114 A pc ^ 118 => v TO c 7^ done 

mC6: pc > 114 => i m c < E(index). size 

mC7: pc = 118 => H(index).table[i m c] = done 

mC8: pc> 110 A fc < H(index).size A k toBeMoved 

=> H(index). table [&] = done 
mC9: pc > 110 A index = currlnd A toBeMoved = 

A k < H(index) .size 

=> H(index).table[fc] = done 
mC10: pc > 116 A val(v mC ) ^ null 

A H(index). table [i m c] = done 

=> H(i mis ).table[ifey(a,H(z ml9 ).size, 0)] 7^ null, 

where a = ADR(v m c) 
mCll: pc > 116 A H(index).table[z m c'] 7^ done 

=> vai(i; m c) = vaJ(H(indcx).table[i m c]) 
A oJdp(H(indcx).table[z m c]) 
mC12: pc > 116 A index = currlnd A val(v m c) 7^ null 

=> vai(w mC ) = val(Y[i mC }) 

Invariants concerning procedure moveElcment (120. . . 126): 

mEl: pc > 120 => vaJ(w mC ) = w mB 

mE2: pc > 120 => w m£ ; 7^ null 

mE3: pc > 120 =4> to = H(i mis ) 

mE4: pc > 121 => a m£ ; = ADR(v mC ) 

mE5: pc > 121 => m mE = to. size 

mE6: pc G {121, 123} => ^6 m£; 

mE7: pc = 123 => k mE = key(a mE , to. size, n mE ) 

mE8: pc > 123 => fc m £ < H(i mi9 ).size 

mE9: pc = 120 

A to.table[Jcey(ACvR(i; m £), to. size, 0)] = null 

=>■ index = currlnd 
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mElO: pc G {121, 123} 

A to.ta.ble[kcy(a m E , to. size, n mE )] = null 
=>■ index = currlnd 
mEll: pc G {121, 123} A per = 103 

A to.table[_kej(a m £;, to. size, n mE )] = null 
=>- indcx.r ^ currlnd 
mE12: pc G {121, 123} A next(currlnd) ^ A to = H(next(currlnd)) 

=>• n m E < H(next(currlnd)).size 
mE13: pc G {123, 125} A w mE + null 

ADR(w mE ) = ADR(to.table[k mE ]) 
V to.table[k mE ] G {del, done} 
mEU: pc > 123 A w mE ^ null 

H(i mi g).table[fc m£ ;] 7^ null 
mE15: pc = 117 A val(v m c) ^ null 

V pc G {121,123} A n mE > 
Vpc = 125 

/i.table[Jccj(AD_R(u mC ), ft,. size, 0)] ^ null, 
where ft = H(i miff ) 
mEI6: pc G {121, 123} 

V (pc = 125 A -n6 mB 

A (val(w mE ) = null V a mE ^ ADR(w mE ))) 
Vm < n mE : 

-iFind(to.table[key(a mE , to. size, m)],a mE ) 
Invariants about the integer array prot. 

prl: prot[i] = (J(prSetJ (i)) + |t(prSet2(i)) + j}(currlnd = i) 

+ (J(next(currlnd) = i) 
pr2: prot [currlnd] > 

pr3: pc G [1, 58] V pc > 62 A pc ^ 65 =>- prot[index] > 
pr4: next(currlnd) 7^ =>■ prot[next(currInd)] > 
pr5: prot[i] = =>- Heap(H[«]) = _L 

pr6: prot[i] < <&(prSet3 (i)) A busy[i] = => Heap(H[i]) = _L 
pr7: pc G [67,72] => prot[i rA ] > 
prS: pc G [81,84] =4> prot[i„ T ] > 
pr9: pc > 97 => prot[i mig ] > 

prlO: pc G [81,82] prot[i„ r ] = ft(pr5eM(i„ T )) + 1 
Invariants about the integer array busy. 

bul: busy[i] = jJ(6uSetl(i)) + $(buSet2(i)) + j}(currlnd = i) 

+ [](next(currlnd) = i) 
bu2: busy[currlnd] > 
bu3: pc G [1,58] 

V pc> 65 A -i(i rA = index A pc G [67, 72]) 
=4> busy[index] > 

bu4: next(currlnd) 7^ =>• busy[next(currlnd)] > 
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hu5: pc = 81 =^> busy[z„r] = 
hu6: pc > 100 => busy[i m i g ] > 

Some other invariants we have postulated: 

Otl: X(0) = null 

Ot2: X(o) ^ null => AGR(X(a)) = a 

The motivation of invariant (Otl) is that we never store a value for the address 
0. The motivation of invariant (Ot2) is that the address in the hash table is 
unique. 

Ot3: return gA = {1, 10, 20, 30, 36, 46, 51} A return rA = {0, 59, 77, 90} 
A return ref = {10,20,30,36,46,51} A return nT = {30,46} 

Ot4: pc e {0, 1, 5, 6, 7, 8, 10, 11, 13, 14, 15, 16, 17, 18, 20, 
21, 25, 26, 27, 28, 30, 31, 32, 33, 35, 36, 37, 41, 
42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 57, 59, 60, 
61, 62, 63, 65, 67, 68, 69, 70, 71, 72, 77, 78, 81, 
82, 83, 84, 90, 94, 95, 97, 98, 99, 100, 101, 102, 
103, 104, 105, 110, 111, 114, 116, 117, 118, 120, 
121, 123, 125, 126} 

B Dependencies between invariants 

Let us write u (p from ipi, ■ ■ ■ , tp n " to denote that ip is proved to be an invariant 
using that ipi, ip n hold. We write 'V -4= ip\,- ■ ■ ,ijj n " to denote that 
predicate ip is implied by the conjunction of ipi, . . . , il> n . We have verified the 
following "from" and relations mechanically: 

Col from filO, Ot3, fil 

Co2 from dc5, Ot3, de6, del, dell 

Co3 from in5, Ot3, in6, inl, inll 

Cnl from Cn6, Ot3 

Cn2 from Cn8, Ot3, del 

Cn3 from CnlO, Ot3, inl, in5 

Cn4 from Cnll, Ot3 

Nol 4= No2 

No2 from nTl, Hc2, rA2, Ot3, Ha2, Hal, rAl, rA14, rA3, nT14, rA4 
Hel from Hal 

He2 from Ha3, rA5, Hal, Hel, rA2 

He3, He4 from Ot3, rA6, rA7, mil2, rAll, rA5 

Hc5 from Hel 

He6 from rA8, Ha3, mi8, nT2, rA5 
Hal from true 
Ha2 from Hal 
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Ha3 from Ha2, Hal, He2, Hel 

Ha4 <= Ha3, He3, He4 

Cn5 from Cn6, Ot3 

Cn6 from Cn5, Ot3 

Cn7 from Cn8, 0t3, del 

Cn8 from Cn7, Ot3 

Cn9 from CnlO, Ot3, inl, in5 

CnlO from Cn9, Ot3, in5 

Cnll from Cnll, Ot3 

Cul from Ot3, Ha4, rA6, rA7, nT13, nT12, Ha2, He3, He4, rAll, nT9, nTIO, 

mil3, rA5 
Cu2 <= Cu6, cu7, Cu3, Hc3, Hc4 
Cu3 from rA6, rA7, nT13, nT12, mi5, mi4, Nc8, rA5 
Cu4 from del, inl, asl, rA6, rA7, Ha2, nT13, nT12, Ne9, Cu9, CulO, de7, 

in7, as5, He3, He4, mi5, mi4, Ot3, Ha4, de3, mi9, milO, dc5, rA5 
Cu6 from Ot3, rA6, rA7, Ha2, nT13, nT12, Ha3, in3, as3, Ne23, mi5, mE6, 

mE7, mElO, mE3, Ne3, mil, mi4, rA5 
Cu7 from Ot3, rA6, rA7, Ha2, nT13, nT12, Ha3, in3, as3, in5, mi5, mE6, 

mE7, mElO, mE3, Nc3, mi4, de7, in7, as5, Ne22, mi9, milO, rA5, He3, 

mil2, mil, Cu9, del inl, asl 
Cu8 from Cu8, Ha2, nT9, nTIO, rA6, rA7, mi5, mi4, mC2, mC5, He3, 

Hc4, Cul, Ha4, mC6, mil6, rA5 
Cu9, CulO from rA6, rA7, nT13, nT12, Ha2, He3, He4, Cul, Ha4, de3, in3, 

as3, mE3, mi9, milO, mElO, mE7, rA5 
Cull, Cul2 from Cu9, CulO, Cul3, Cul4, del, inl, asl, rA6, rA7, Ha2, nT13, 

nT12, He3, He4, Cul, Ha4, in3, as3, mil4, mil5, de3, inlO, as8, mil2, 

Ot2, 65, de8, in8, as6, Cul5, dell, inll, rA5 
Cul3, Cul4 from He3, He4, Ot2, del, inl, asl, Otl, rA6, rA7, nT13, nT12, 

Ha2, Cu9, CulO, Cul, Ha4, de3, in3, as3, Cull, Cul2, inlO, as8, 65, de8, 

in8, as6, Cul5, mil7, mil8, mil2, mi4, dell, rA5 
Cul5 from He3, He4, rA6, rA7, nT13, nT12, Ha2, Cul, Ha4, del, inl, asl, 

de3, in3, as3, 65, de8, in8, as6, mil2, mil9, mi4, Ot2, Cu9, CulO, Cull, 

Cul2, Cul3, Cul4, rA5 
Cul6 4= Cul3, Cul4, Cul5, He3, He4, Otl 
Ncl from nT9, nTIO, mi7 
Ne2 from Nc5, nT3, mi8, nT9, nTIO 
Ne3 from Nel, nT9, nTIO, mi8 
Ne4 from Ncl, nT9, nTIO 
Ne5 from Ot3, nT9, nTIO, mi5 
Ne6 NelO, Ne24, He6, He3, He4, Cu4 

Ne7 from Ha3, rA6, rA7, rA8, nT13, nT12, nTll, He3, He4, mi8, nT7, 

Ne5, Ha2, He6, rA5 
Ne8 from Ha3, rA8, nTll, mi8, nT6, Ne5, rA5 

Ne9 from Ha3, Ha2, Nc3, Ne5, de3, as3, rA8, rA6, rA7, nT8, nTll, mC2, 

nT4, mi8, rA5 
Ne9a from Ha3, Ne3, rA5, de3, rA8, nT4, mi8 
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NelO from Ha3, Ha2, de3, rA8, nTll, Ne3, He6, mi8, nT8, mC2, nT2, Ne5, 
rA5 

Nell from Ha3, Ha2, He6, nT2, nT8, rA8, nTll, mi8, Nc3, mC2, rA5 

Nel2, Ncl3 from Ha3, Ha2, Cu8, He6, He3, He4, Cul, dc3, in3, as3, rA8, rA6, 
rA7, nTll, nT13, nT12, mil2, mil6, mi5, mi4, dc7, in7, as5, Ot2, dcl,inl, 
asl, Cu9, CulO, Cul3, Cul4, Cul5, as9, 65, de8, in8, as6, mC2, Ne3, Otl, 
Nel4, Ne20, mE16, mE7, mE4, mEl, mE12, mE2, Nel5, Nel6, Nel7, 
Nel8, mi20, dell, inll, rA5 

Ncl4 from Ha3, Ha2, He6, He3, He4, nT2, nT8, de3, in3, as3, rA8, nTll, 
Ot2, del, inl, asl, Cu9, CulO, mi8, Nc3, mC2, mE7, mE16, mEl, mE4, 
mE12, Nel7, Nel8, Cul, rA5 

Nel5, Nel6 from Ha3, Ha2, Cu8, He6, He3, Hc4, Cul, de3, in3, as3, rA8, rA6, 
rA7, nTll, nT13, nT12, mil2, mil6, mi5, mi4, dc7, in7, as5, Ot2, del, inl, 
asl, Cu9, CulO, Cul3, Cul4, Cul5, as9, 65, dc8, in8, as6, mC2, Ne3, Otl, 
Nel9, Ne20, Nel2, Nel3, mE16, mE7, mE4, mEl, mE12, mE10, mE2, 
inll, dell, rA5 

Nel7, Nel8 from Ha3, Ha2, mi8, He6, He3, He4, Cul, nT2, de3, in3, as3, rA8, 
rA6, rA7, nTll, nT13, nT12, dc7, in7, as5, Ot2, del, inl, asl, Cu9, CulO, 
nT8, mE2, 65, de8, in8, as6, mC2, Ne3, mCll, mC6, mC12, mE7, mElO, 
mEl, Cu8, Cul5, Cul3, Cul4, Cull, Cul2, as8, dell, rA5 

Nel9 from Ha3, Ha2, He6, nT2, nT8, de3, in3, as3, rA8, nTll, mi8, Ne3, 
mE7, Ncl4, mE16, Otl, mEl, mE4, mE12, Nel7, Nel8, rA5 

Ne20 from Ha3, Ha2, Cu8, Hc6, Hc3, Hc4, Cul, Ha4, de3, in3, as3, rA8, rA6, 
rA7, nTll, nT13, nT12, mil2, mil6, mi5, mi4, Nel, de7, in7, as5, del, inl, 
asl, Cu9, CulO, Cul3, Cul4, Cul5, as9, 65, dc8, in8, as6, mC2, Ne3, Otl, 
mi20, inll, rA5 

Ne22 from Ot3, rA8, Ha2, nTll, Ha3, de3, in3, as3, mi5, mi4, Nc3, nT18, 

mE3, mi8, mElO, mE7, mE6, Ne5, nT5, nT2, rA5, nT8, nT12, mC2, mE2 
Nc23 <= Cu6, cu7, Ne6, Ne7, He3, He4, Ne22, He6 
Ne24 Ne27, He6 
Ne25 <= Nel9, Nel7, Nel8, He6 
Ne26 <= Ncl7, Nel8, Hc6 
Ne27 <^ Cul6, Nc25, Ne26, Nel 7, Nel8, He6 
61, del, inl, asl from 

62 from 62, Ot3 

63 from 64, Ot3, rA6, rA7, Ha2, rA5 

64 from Ot3, rA6, rA7, nT13, nT12 

65, de8, in8, as6 <= Cu2, delO, inlO, as8, 68, He3, Hc4 

66 from Ot3, 61, del, inl, asl, rA6, rA7, Ha2, nT13, nT12, mi9, milO, Cu9, 
CulO, He3, He4, Cul, Ha4, 64, in3, as3, rA5 

67 from 68, 66, 62, Ot3, 61, del, inl, asl, rA6, rA7, Ha2, nT13, nT12, mi9, 
milO, Cu9, CulO, Hc3, Hc4, Cul, Ha4, 64, in3, as3, rA5 

68 from 64, 67, 62, Ot3, 61, del, inl, asl, rA6, rA7, Ha2, nT13, nT12, mi9, 
milO, Cu9, CulO, Hc3, Hc4, Cul, Ha4, in3, as3, rA5 

69 <= Cul, Ha4, Cu9, CulO, Cull, Cul2, 68, 63, 64, 65, dc8, in8, 
as6, He3, He4 
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filO from 69, Ot3 

fill, del2, inl2, aslO from Ot3, nT9, nTIO, mi9, milO, Cu8, 64, dc3, in3, 

as3, 63, de2, in2, as2 
dc2 from dc3, Ot3, rA6, rA7, Ha2, rA5 
de3 from Ot3, rA6, rA7, nT13, nT12 
dc4, in4, as4 from Ot3 
de5 from Ot3 
de6 from Ot3, del, dell 

de7, in7, as5 <= de3, in3, as3, Cul, Ha4, del3, inl3, asll 

de9 from Ot3, del, inl, asl, rA6, rA7, Ha2, nT13, nT12, mi9, milO, 

Cu9, CulO, de3, de7, in7, as5, rA5 
delO from de3, de9, Ot3, del, inl, asl, rA6, rA7, Ha2, nT13, nT12, mi9, 

milO, Cu9, CulO, dc7, in7, as5, He3, He4, rA5 
dell <= delO, de2, de3, He3, He4, Cul, Ha4, Cu9, CulO, Cull, Cul2, 65, 

de8, in8, as6 

del3, inl3, asll 4= Ax2, de2, de3, de4, in2, in3, in4, as2, as3, as4 

in2 from in3, Ot3, rA6, rA7, Ha2, rA5 

in3 from Ot3, rA6, rA7, nT13, nT12 

in5 from Ot3 

in6 from Ot3, inl, inll 

in9 from Ot3, del, inl, asl, rA6, rA7, Ha2, nT13, nT12, mi9, milO, Cu9, 

CulO, Hc3, He4, in3, de7, in7, as5, rA5 
inlO from in9, 62, Ot3, del, inl, asl, rA6, rA7, Ha2, nT13, nT12, mi9, milO, 

Cu9, CulO, He3, He4, in3, de7, in7, as5, rA5 
inll <= inlO, in2, in3, Cul, Ha4, Cu9, CulO, Cull, Cul2, 65, dc8, in8, as6 
as2 from as3, He3, He4, Ot3, rA6, rA7, Ha2, rA5 
as3 from Ot3, rA6, rA7, nT13, nT12 

as7 from Ot3, del, inl, asl, rA6, rA7, Ha2, nT13, nT12, mi9, milO, Cu9, 

CulO, as3, de7, in7, as5, rA5 
as8 from as7, Ot3, del, inl, asl, rA6, rA7, Ha2, nT13, nT12, mi9, milO, Cu9, 

CulO, He3, He4, as3, de7, in7, as5, rA5 
as9 as8, as2, as3, He3, He4, Cul, Ha4, Cu9, CulO, Cull, Cul2, 65, de8, 

in8, as6 
rAl from Ha2 
rA2 from Ot3 

rA3 from Ot3, rA9, Hc2, Hel, rA2, rAl3 

rA4 from Ot3, nT14 

rA5 from Ot3, rAl, rA2, Ha3, He2 

rA6, rA7 from Ot3, nT13, nT12, nT14, rAll, mi4, bu2, bu3, Ha3, mi6, Ha2, 

He3, He4, He2, rA2 
rA8 from Ot3, bu4, nT14, mi6, Ne2, mi5 
rA9 from Ot3, Ha2, nT14, Hel, He2 
rA10 from Ot3 

rAll from Ot3, nT13, nT12, mi2 
rA12 from Ot3, nT9, nTIO 
rA13 from Ot3, rA5 
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rA14 from Ot3, rA4, Hel, rA2 

nTl from Ot3, pr5, Ha3, nT14, nT16, Ha2 

nT2 from 0t3, nT14, Ha3, rA5 

nT3 from 0t3, nT9, nTIO 

nT4 from Ot3, Ha3, dc3, nT13, nT12, nT15, rA5 

nT5 from Ot3, Ha3, in3, as3, nT13, nT12, nT15, nT18, mE3, mi4, rA5 
nT6 from Ot3, nT13, nT12, nT14, Ha3, rA5 

nT7 from 0t3, nT13, nTl 2, nT15, rA6, rA7, Ha2, mi9, milO, nT14, Ha3, 
nT16, rA5 

nT8 from Ot3, de3, in3, as3, nT13, nT12, nT15, nT18, mE3, mi4, Ha3, mC2, 

nT16, nT2, Ha2, rA5 
nT9, nTIO from Ot3, pr2, pr3, nT18 
nTll from Ot3, pr4, nT16, mi8 
nT13, nT12 <= nT9, nTIO, Ha3, Hc3, Hc4 
nT14 from Ot3, nT9, nTIO, nT18, nT16, pr7 
nT15 4= nT14, Ha3, nT2 
nT16 from Ot3, pr8 
nT17 from Ot3, mi5, pr4, nTll, milO 
nT18 from Ot3, pr9, mi5, nTll 
mil from Ot3, mi9, milO, milO 
mi2 from Ot3, Ne4 

mi3 from Ot3, fill, dcl2, inl2, aslO, nT9, nTIO, Ne5 

mi4 from Ot3, mi9, milO, mi3 

mi5 from Ot3, nT9, nTIO, Nc5, milO, mi4 

mi6 from Ot3, mi5, bu6, rA8, mi9, milO, bu4, mi4 

mi7 from Ot3, mi2, mi7, mi4, nT18, Nc2, milO, nT17, mi3 

mi8 from Ot3, milO, Nc2, mi3 

mi9, milO from Ot3, He3, He4, nT9, nTIO, nT18, Nc3, Ha3, mi3, nT17, 

milO, He2, mi4, mil2, mi6, He6 
mill from Ot3, nT18, mi9, mi6, mi6 

mil2 from Ot3, rA8, nT2, He6, mi9, mi5, mi3, Ha3, mi4, rA5 
mil2 from Ot3, mil 2, nT18, mi6, Ha3, mi4, rA5 

mil3 from Ot3, rA6, rA7, Ha2, nT13, nT12, He3, He4, mi9, milO, mC9, rA5 
mil4, mil5 <= Nel2, Ncl3, mi5, Cul5, mil3, Ot2, He3, He4, Nel7, Nel8, 

Cu8, Hc6, Hc5, mi4, Otl 
mil6 4= Nell, mi5, mi4 

mil7, mil8 <= Nel5, Nel6, mi5, Cul5, mil3, Ot2, Hc3, Hc4, Nel7, Ncl8, 

Cu8, He6, He5, mi4 
mil9 <^= Ne20, mi5, Cul5, mil3, Ot2, Hc3, Hc4 

mi20 from Ha3, Ha2, Cu8, He6, He3, He4, Cul, Ha4, de3, in3, as3, rA8, 
rA6, rA7, nTll, nT13, nT12, mi5, mi4, dc7, in7, as5, Ot2, del, inl, 
asl, Cu9, CulO, Cul3, Cul4, Cul5, as9, fi5, de8, in8, as6, mC6, Ne3, 
Ot3, mCll, mil3, mi9, milO, mC2, mE3, mE10, mE7, mC12, mEl, 
mE13, Nel7, Nel8, mE2, mE4, Otl, mE6, NclO, inll, rA5 

mCl from Ot3, mi6, mill, nT18 

mC2 from Ot3, rA6, rA7, nT13, nT12, mC2 
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mC3 from Ot3, mC3, nT13, nT12, rA6, rA7, Ha2, rA5 

mC4 from Ot3, mC4, mC2, mC3, He3, Hc4, rA6, rA7, Ha2, rA5 

mC5 from Ot3 

mC6 from 0t3, rA6, rA7, Ha2, nT13, nT12, mC2, rA5 
mC7 from 0t3, rA6, rA7, Ha2, nT13, nT12, mC2, rA5 
mC8 from 0t3, rA6, rA7, Ha2, nT13, nT12, He3, He4, mC7, rA5 
mC9 from 0t3, rA6, rA7, Ha2, nT13, nT12, He3, He4, mi9, milO, He5, 
mC7, mC8, rA5 

mClO from Ot3, rA6, rA7, Ha2, nT13, nT12, mC2, del, inl, asl, mi6, 

Ha3, mi4, nT18, mE15, mCll, mi5, rA5 
mCll from Ot3, rA6, rA7, Ha2, nT13, nT12, mC2, rA5 
mC12 from Ot3, rA6, rA7, mC2, mCll, Cu9, CulO, dc7, in7, as5, mi9, 

mC6 
mEl from Ot3 
mE2 from Ot3 

mE3 from mCl, 0t3, mi6, nT18 
mE4 from 0t3, mEl 

mE5 from 0t3, mE3, Ha3, mi6, mi4, nT18, Ha2, rA5 
mE6 from 0t3 

mE7 from 0t3, Ha2, Ha3, mi6, mi4, mE3, rA5 

mE8 from 0t3, Ha3, mi6, mi4, nT18, Ha2, mE3, rA5 

mE9 from Cul, Ha4, 0t3, Ha2, Ha3, mi6, mi4, mE3, mC2, mClO, mEl, 

mCl, del, inl, asl, mil3, mil2, mC6, mE2, rA5 
mE10 from del, inl, asl, mE3, mi6, Ot3, Ha2, Ha3, mi4, mEll, mE9, 

mE7, rA5 

mEll 4= mE10, mi 13, mE16, mil6, mi5, mE3, Ncl2, Nel3, mC12, mE2, 

mEl, mE4, mC6, mE12, mil2, Cul3, Cul4, He3, Hc4, mi4 
mE12 4= Ne23, Ne22, mE16, He6, Ne8 

mE13 from Ot3, Ha2, mE14, del, inl, asl, Ha3, mi6, mi4, mE3, rA5 
mE14 from Ot3, Ha2, del, inl, asl, Ha3, mi6, mi4, nT18, mE3, mE2, rA5 
mE15 from Ot3, mEl, Ha2, del, inl, asl, Ha3, mi6, mi4, nT18, mE3, mE2, 

mE7, mE14, mE4, rA5 
mE16 from Ha3, Ha2, mE3, del, inl, asl, mi6, mE2, mE4, mEl, mE7, mi4, 

Ot3, mE14, mE13, rA5 
prl from Ot3, rAll, rA10, nT9, nTIO, Nc5, mi2, mi4, mi8, mi5 
pr2, pr3 from prl, Ot3, rAll, mil 
pr4 4= prl 
pr5 4= pr6, prl, bul 

pr6 from Ot3, Ha2, nT9, nTIO, nT14, nT16, He2, rA2, prl, bul, prlO, 

rA9, Hel, rA4 
pr7, pr8, pr9 4= prl, mi4 
prlO from Ot3, prl, nT9, nTIO, nT14, nT17 
bul from Ot3, rAll, rA10, nT9, nTIO, Nc5, mi2, mi8, mi5, bu5 
bu2, bu3 4= bul, Ot3, rA10 
bu4 4= bul 

bu5 from Ot3, nT9, nTIO, nT16, nT18, prl, bul 
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bu6 <= bul, mi4 
Otl from del, inl, asl 
Ot2 from del, inl, asl 
Ot3 from true 
Ot4 from Ot3 
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